"Lake of Fire," currently on limited release in the United States, unwinds over more than two and a half hours of interviews with some of the leading figures from the pro-life and pro-choice camps.
But it is the graphic and disturbing depiction of termination procedures, filmed like the rest of the movie in black and white, that marks the film out.
"From the moment I started making the film I thought I have to show an abortion, which at the time had never been done before," Kaye, best known for his 1998 neo-Nazi feature "American History X," told AFP in an interview.
"There was no question about whether or not that was the right thing to do because if I'm documenting two sides of the argument, that is one side of the argument and you have to show it," he said.
Kaye, who cut his teeth directing commercials and music videos, separately admits to being "addicted to controversy when I began."
One scene depicts a doctor sifting through a surgical tray after performing a late-term abortion, where the grisly residue of an arm, a foot and part of a face can be clearly made out.
"It's about as shocking as any motion picture can ever get. It's illegal to film someone being killed," said Kaye.
They may be the kind of images used by anti-abortion activists, but Kaye also doesn't shy from showing pictures of a kneeling and bent-over naked woman who died after performing a botched abortion on herself with a wire coat hanger.
Kaye worked for more than 15 years on "Lake of Fire" -- anti-abortion activist John Burt's description of the hell awaiting abortionists -- and said his goal when he set out was simply to show both sides of the argument.
"The concept was to make a film about the debate over the issue of abortion but to make it a non-propagandist way and to create a kind of war of words."
He said he wanted "to create this kind of a weave where we really explore the issue without taking any sides."
"It's very easy for me to do that because... I don't really have a point of view. I'm not a politician or a commentator," he said.
The film features leading thinkers including linguist and leftist intellectual Noam Chomsky, who dismisses the notion of certainty in the debate.
"You're not going to get the answers from holy texts. You're not going to the answers from biologists," he says. "These are matters of human concern."
"There are conflicting values and taken in isolation each of these values is quite legitimate," he adds. "Choice is legitimate, preserving life is legitimate."
If the film arrives at any conclusion it is neatly summed up by Harvard University law professor Alan Dershowitz when he says: "Everybody is right when it comes to the issue of abortion."
"In the end human beings have to decide. In the end each of us has to decide using whatever resources we have available to us: religion, our mind, our sense of what is right and wrong in society," he adds.
A key figure in the film is Norma McCorvey, better known as Jane Roe, the lead plaintiff in the landmark Roe vs Wade case that legalized abortion in the United States in 1973 and continues to shape the political landscape.
McCorvey was the poster child of the pro-choice camp until she turned pro-life and explains in the film how anti-abortion campaigners told her that "I was the one that was responsible for all the dead babies."
Even after spending years working on the project, Kaye, however, admits to not knowing where he stands in the debate.
"My position on the subject is that I don't really know what's right. I didn't know much in the beginning... and at the end I was just as confused."