As for the message, it should vanish after apply settings for the first time if I'm not mistaken, unfortunately we can't see the difference between improperly configured and not run yet because it results in the same situation (the file not being there).
yes, you can check the contents of the field, unfortunately it doesn't tell you anything about the validity of the data (in which case the error message probably is still fine, surely we could replace "a source" for "a valid source", but that wouldn't tell you more than it already did).
The apply button itself is synchronous if I'm not mistaken, when the spinner stops it should end up with a correct state when the provided url was valid.