I do not remember what the settings are right off, but have you tried changing the file format for images in the export settings? I know that some platforms, like mobile, do not support all of the compression file formats Godot offers and trying to export a project with those file formats leads to the textures not loading, being corrupt, or otherwise having issues. It sounds like something similar to that is happening here.
I would see if changing the texture settings fixes the issue. It should be part of the export settings for HTML, if I am remembering correctly.