Object Dither works with binary values rather than smooth scalar values but it still can achieve that semi transparent look, sort of, by controlling the amount of pixels visible. Look up examples of dithering I guess.
I'm not sure if you can set a limit to how far it can disappear so yes there is a chance it might make the wall completely transparent but I think that would be the case with Distance fade either ways. That is to say it's not specific to object dither.
You could try having 2 materials on your wall though, one that is visible most of the time with the distance fade property making it fade under specific conditions and a 'bottom layer' that gets revealed by the distance fade that is always set to a specific transparency value.
Note that this will mean at least 2 draw calls per each wall section and will have a performance impact to some degree.