Evaluating Silicon Sampling: LLM Accuracy in Simulating Public Opinion on Facial Recognition Technology

Large Language Models (LLMs) have demonstrated remarkable capabilities in mimicking human behaviors, leading to growing interest in their potential for survey research through “silicon sampling”, a method where LLMs generate responses when prompted with personas. This study evaluates the effectiveness of silicon sampling in emerging technology acceptance by simulating public opinion on facial recognition technology (FRT). I compare LLM-generated responses against actual survey data from 6,076 respondents across China, Germany, the United Kingdom, and the United States. I test three LLMs (GPT-4o, Claude 3.5 Sonnet, and DeepSeek V3) under three prompting conditions: demographic information only, contextual information only, and a combination of both. Performance was assessed using Mean Absolute Error (MAE) and Quadratic Weighted Cohen’s Kappa (QWK) metrics. Results demonstrate that demographic-only prompting yields poor simulation accuracy (MAE: 0.90-1.73; QWK: near zero), while incorporating contextual information about FRT experiences and perceptions significantly improves performance. The optimal approach combining demographics with contextual information achieved QWK scores of 0.40-0.45, indicating moderate agreement with human responses. While silicon sampling cannot precisely replicate individual-level survey responses, the findings suggest it holds promise as a complementary tool for survey research, particularly in early research stages. This study provides practical guidance for researchers employing LLMs in surveys and highlights the importance of contextually relevant prompts for effective silicon sampling.