Be a little smarter about qual handling for semi-joins: a qual that mentions

only the outer side can be pushed down rather than having to be evaluated
at the join.