Keep storing large images, just don’t index the binary data itself!

I’ve got the question a lot of times: An app using Dexie.js is working fine for a while and then after a while the IndexedDB database has become slower and finally crashes. In almost all these cases, the mistake made is indexing large binary data or large Base64 strings representing binary data such as image data, songs or movies.

const db = new Dexie("myImageDB");
db.version(1).stores({
  images: '++id, imageData' // please don't!
});

A common misunderstanding seem to be that all properties should be indexed. Yes, IndexedDB can store movies, songs and images, but that doesn’t mean you should index them. There is normally no gain of indexing a picture, tune or movie because how would you use it in a where-clause?

db.songs.where('waveBinary').equals(x) // ?
db.songs.where('largeMovieData').between(a, b) // ??

I don’t think there are any good example of querying those large indexes. An equals-query could be valid, but then it’s better to store a hash of the data and index and query that hash property instead:

db.songs.where('songHash').equals(x)

Other good strategies of indexing image data is to have an alternate field along with the data. For example, if the image is analyzed by an AI engine, you could index the output of tags that the AI engine would see in the image, such as “cat”, “fur”, “water”, “spring” etc. Image searches could be done by querying the tags property in instead to find images of a “cat” etc.

const db = new Dexie("myImageDB") as Dexie & {
  images: EntityTable<ImageRow, 'id'>
};

db.version(2).stores({
  images: '++id, *tags, imageHash'
});

interface ImageRow {
  id: number
  imageData: Blob // stored by not indexed!
  imageHash: UInt8Array // MD5 digest
  tags: string[] // From image AI analysis, "cat", "water", etc.
}

IndexedDB does not put any limit on the size of the indexed field. I would have wished that the engine would throw when trying to store a too large entry in an indexed property rather than mysteriously get slower and slower and finally crash later on. I think I’m gonna put a limit of at least 2000 bytes in Dexie to overcome this issue with IndexedDB and help customers avoid this common mistake in the future.

Most enterprise database engines do not allow indexing columns larger than ~1000–2000 bytes including Microsoft SQL Server and PostgreSQL. I think we should introduce the same in IndexedDB. At least, I want to do it in Dexie.

Keep storing large images, just don’t index the binary data itself! was originally published in Dexie.js on Medium, where people are continuing the conversation by highlighting and responding to this story.