Whose doctor does the AI recommend? An algorithm audit of reputation and demographic signals in large language model-assisted physician choice
Ranking
Overall
79
Content
95
Popularity
41
Observed public metrics from 1 member.
Merged summary
TL;DR - A randomized audit of seven LLMs found that physician recommendations are driven mainly by ratings and fees, while also exhibiting smaller demographic and positional biases that their explanations rarely disclose. This supports recurring behavioral audits over reliance on model self-reporting.
- Across 40,068 scored responses, increasing a physician’s rating from 3.9 to 4.7 raised selection probability by 31.4 percentage points.
- Raising the visit fee from $90 to $190 reduced selection probability by 20.0 points.
- Female-, Hispanic-, South-Asian-, and Black-signaled names received selection advantages of 1.3–2.9 points over comparison groups.
- Models mentioned gender or ethnicity in at most 0.03% of explanations, making these measured effects effectively invisible through self-reporting.
Sources (1)
Whose doctor does the AI recommend? An algorithm audit of reputation and demographic signals in large language model-assisted physician choice
Public signals
Semantic Scholar citations 0 · Semantic Scholar influential citations 0
TL;DR - A randomized audit of seven LLMs found that physician recommendations are driven mainly by ratings and fees, while also exhibiting smaller demographic and positional biases that their explanations rarely disclose. This supports recurring behavioral audits over reliance on model self-reporting.
- Across 40,068 scored responses, increasing a physician’s rating from 3.9 to 4.7 raised selection probability by 31.4 percentage points.
- Raising the visit fee from $90 to $190 reduced selection probability by 20.0 points.
- Female-, Hispanic-, South-Asian-, and Black-signaled names received selection advantages of 1.3–2.9 points over comparison groups.
- Models mentioned gender or ethnicity in at most 0.03% of explanations, making these measured effects effectively invisible through self-reporting.