AI Images Often Can't Be Traced Back to a Single Training Picture

Pictures made by AI image generators trained on large datasets often cannot be traced back to any individual training image, according to a study published Aug. 18 in Nature Communications by Zheng Dai and David K. Gifford of MIT's Computer Science and Artificial Intelligence Laboratory.

The pair trained 24 ensembles of diffusion models, the design behind most current image generators, on subsets of seven public image collections ranging from 256 to 162,770 images, the paper states. The ensembles were built so that the influence of a single training image could be switched off without retraining the whole model, which let the authors ask what a given output would have looked like had that image never been in the data.

"If you take away a piece of data and the output of the model doesn't change, then that piece of data didn't affect the output," Dai said in MIT's account of the work.

The paper reports an inverse power law between the number of images a model was trained on and how far its outputs shift when training data is removed, a distance the authors call the counterfactual radius. The fits hold at R-squared of 0.85 for a geometric version of that distance and 0.59 for a semantic version, at p below 10 to the minus 5 for both relations, according to the paper.

The authors also tested attribution methods that work by hunting for training images resembling the output. Ensembles trained on more data produced larger false attribution rates, the paper reports, meaning a similar-looking training image was named as an influence when deleting it in fact changed nothing. Each rate was estimated from 100 samples.

To check that the switch-off design was not itself producing the effect, the researchers repeated the experiment at small scale by brute force, training 1,282 separate models, MIT's release says.

The collections used were MNIST, Fashion-MNIST, CIFAR-10, CIFAR-100, CelebA, MetFaces and ArtBench. The paper is open access under a Creative Commons Attribution 4.0 license, published as article 6974 in volume 17.