Your neighbor is correct it is illegal to discharge an air gun within city limits.
The best thing to fire in safe direction so he has nothing to hold against you are well within city limits then you are well within your within your local sheriffs office and ask them if your neighbor to fire in safe direction.
The time you gust need backstop and safety gear.