Worry about it if it is actually in the way, not just appears to be. Depends on which satellites you are trying to see, but bear in mind that in the Southern UK, most of the watchable satellites are at 25 degrees elevation or above, which is about 1 in 2. So a 30ft obstacle, if more than 60 ft away, will not cause a problem.
PS. Don't know where you are, as you don't say, just using Southern UK as a reference.