Bothered by the frequent talk about the number of Catholics on the Supreme Court, Justice Samuel Alito voiced his concerns in a recent speech to an Italian-American law group in Philadelphia, reported The Associated Press.

“There has been so much talk lately about the number of Catholics serving on the Supreme Court,” said Alito. “This is one of those questions that does not die.”

Alito voiced his concern about “respectable people who have seriously raised the questions in serious publications about whether these individuals could be trusted to do their jobs.”

According to Alito, the U.S. Constitution settled the question long ago, with its guarantee of religious freedom.

Alito is one of six justices on the nine-member court who were raised Catholic. A dozen of the court’s past 111 jurists have been Catholic.