.Attributes Medicine, Posted online: 26 July 2024 doi:10.1038/ s41591-024-03181-6Although sizable foreign language versions (LLMs) present commitment in measured environments, a study now reveals their constraints in real-world professional applications and also points the way towards durable analysis and benchmarking just before clinical use.