A person who wrote to the panel said it could lead to “unconscious bias” against some groups of people.
The panel, largely made up of local councillors, advises and scrutinises the police and crime commissioner.
Panel chairman Alun Lloyd Jones said they represented the make-up of local councils.
He said the authorities would be asked to reflect diversity in their nominations.
The panel consists of 12 councillors nominated by Carmarthenshire, Ceredigion, Pembrokeshire and Powys plus two co-opted members, according to the Local Democracy Reporting Service.
Dyfed-Powys Police serves a largely rural swathe of mid and west Wales, with just over 2% of the population identifying as black or minority ethnic in the 2011 census.
The person who submitted the written question to the panel said they were shocked there was only one “older white woman” serving on it, with the rest being “older white men”.
They called for the panel to be made more “representative of our society”, by considering including younger women, ethnic minorities, and gay members.
“Why was it decided that the panel should be made up of county councillors, because these are overwhelmingly retired, white straight older males,” asked the complainant, whose name was withheld.
Mr Jones told a meeting of the panel on Friday that Home Office regulations required the panels to be politically balanced and, as far as possible, gender balanced and reflective of the area they cover.
“The panel only has a say on co-opted members,” he said, adding that just two out of eight applications for the co-opted posts in 2016 were from women.
Helen Thomas, one of the co-opted members, said she was “not surprised” to hear the complaint.
However, she said she was sure there was no evidence of unconscious bias or discrimination on the panel’s work.
“We probably ought to note the saying ‘don’t judge a book by its cover’,” Ms Thomas added.