The union also wants the governor to fire New York City Mayor Bill de Blasio.
The Police Benevolent Association for the city represents almost 25,000 active members of the NYPD. The union made its demands after its delegate assembly voted unanimously on resolutions of no confidence in both O'Neill and de Blasio.
PBA president Patrick Lynch said in a statement, "The unjust termination of P.O. Daniel Pantaleo was merely the final straw: both men have displayed an appalling pattern of malfeasance and nonfeasance that disqualifies them from continuing to serve in their current offices."
Pantaleo's firing came more than five years after he held Garner in a chokehold while trying to arrest him. Garner later died, and the medical examiner said the choke contributed to his death. Although the Justice Department chose not to pursue charges against Pantaleo, an NYPD administration judge suggested he be terminated.
A spokesperson for de Blasio called the no-confidence vote "another attempt by the PBA to divide our city." Meanwhile, police sources told The Wall Street Journal that O'Neill has no plans to resign as commissioner.