A 10-Year-old CNN Outperforms Frontier LLMs For Skin Cancer Diagnosis

A dermatologist’s comparison of three AI systems found that a fine-tuned Inception v3 convolutional neural network, first published in 2015, correctly classified more biopsy-confirmed skin lesions than two newer large language models. On a test of 200 lesions, the CNN named the correct diagnosis in 125 cases (62%); GPT-6 Astra did so in 84 (42%), and Claude Fable 5.1 in 85 (42%). The result is a limited image-test finding, not evidence that the CNN is ready to diagnose patients or that the language models are unsafe for all clinical uses.

The report, published by a dermatologist on Lynch Lab, says the test used images from MILK10k, a public dataset released through the International Skin Imaging Collaboration in 2025. Each lesion had a clinical close-up and a dermoscopic image. All 200 selected lesions had been biopsied and their diagnoses confirmed by histopathology, including benign lesions that had been removed because a clinician had concerns.

The sample included 30 melanomas, 25 basal cell carcinomas, 25 squamous cell carcinomas, 20 actinic keratoses, 50 moles and 50 seborrhoeic keratoses. The two LLMs received both images and the same prompt, asking for the most likely diagnosis from six options or “unsure,” a confidence level, a second choice, and whether the lesion could be discharged, biopsied or followed up. The report says neither model declined a case.

The CNN was an Inception v3 model fine-tuned on 68,011 clinical and dermoscopic images from the ISIC Archive. The author says the test lesions, other images of those lesions and images from the same patients were excluded from training and validation. The model considered both views, with greater weight given to the dermoscopic image. It achieved higher overall diagnostic accuracy, though results differed by condition: GPT-6 Astra correctly identified all 25 basal cell carcinomas, while the CNN identified 24; for melanomas, the CNN identified 13 of 30, compared with 8 for GPT-6 Astra and 11 for Claude Fable 5.1.

Accuracy Does Not Equal Clinical Readiness

The findings illustrate why performance claims about medical AI depend on the task, the images and the test population. In this comparison, a purpose-trained image classifier had higher overall accuracy than two general-purpose models, but none correctly identified every lesion. The melanoma results are especially relevant because the report describes a case in which both LLMs labeled an invasive melanoma as a benign seborrhoeic keratosis and suggested discharge; the CNN identified it as melanoma from the dermoscopic view.

That example and the aggregate scores should not be read as a head-to-head clinical trial. The test used a small sample selected from a public dataset, and all lesions had already been biopsied. The report does not establish how any model would perform with images from different devices, patients or clinical settings, or whether using it would improve decisions or patient outcomes. The comparison does, however, underline that a model’s age or general reputation is not a substitute for testing on the specific medical task it is meant to support.

How the Image Test Was Run

The CNN was not described as a state-of-the-art system. Its base architecture, Inception v3, was published by Google in 2015 and had originally been pretrained on everyday photographs. The dermatologist fine-tuned all its layers on dermatology images for about 20 minutes on a single H100 GPU, according to the report. The author says the 200 test cases were not used to select the model or set its classification threshold.

The test asked for one of six specific diagnoses, making the reported score a measure of exact diagnostic classification on this dataset. It is distinct from the separate question of whether a lesion should be discharged, biopsied or followed up. The source gives examples of disposition judgments, but the overall accuracy figures provided are for naming the diagnosis. The report also says the small CNN can run in a phone browser using WebGPU; that feature does not itself indicate clinical validation.

“A ten-year-old network, fine-tuned for about twenty minutes, beat both of them at recognising skin cancer.”

— The dermatologist who conducted the comparison

Limits of the 200-Lesion Sample

The report describes a single test of 200 lesions, not a multi-site clinical evaluation. It does not provide evidence here about performance across broader patient populations, skin tones, image quality levels or real-world clinical workflows. The sample was drawn from a public dataset, and the author’s description does not establish whether independent researchers have replicated the comparison.

It is also unclear from the reported results how often each system would recommend discharge, biopsy or follow-up across all cases, or how those recommendations compare with clinicians’ decisions. Overall diagnosis accuracy does not show the clinical consequences of individual errors, and it does not establish that any system can safely replace professional assessment. The report’s scores should be understood within its stated dataset, selection and prompt conditions.

Independent Testing Is the Next Step

The report does not announce a planned clinical trial or a regulatory review. The next useful evidence would be independent testing on larger, varied datasets, with methods that make it possible to check performance across lesion types and patient groups. Evaluations could also separately report diagnostic accuracy and decisions about discharge, biopsy or follow-up.

Until such evidence is available, the published comparison supports a narrow conclusion: on the author’s selected set of 200 biopsy-confirmed lesions, this fine-tuned CNN produced more exact diagnoses than the two tested LLMs. It does not show that the CNN is ready for routine use, or that results will hold outside this test.

Key Questions

Which system scored highest in the test?

The Inception v3 CNN correctly named 125 of 200 lesions (62%). GPT-6 Astra named 84 correctly (42%), and Claude Fable 5.1 named 85 (42%), according to the report.

What kinds of skin lesions were included?

The sample contained six diagnoses: melanoma, basal cell carcinoma, squamous cell carcinoma, actinic keratosis, moles and seborrhoeic keratoses. All 200 lesions had diagnoses confirmed by histopathology.

Does this show the CNN is ready to diagnose patients?

No. The report describes a test on 200 lesions from a public dataset, not a clinical trial or validation in routine care. It does not establish that the model can safely guide patient decisions.

Did the CNN outperform the LLMs on every diagnosis?

No. The CNN had the highest overall score and led on several categories, but GPT-6 Astra correctly classified all 25 basal cell carcinomas, compared with 24 for the CNN. Performance varied by diagnosis.

Source: hn