How to import 16 bit per-channel PNGs?

arakash

I'm doing a open-world style terrain system, which is coming along well. But due to how the HeightMapShape works, I made the Terrain Vertex shader sample from the center of each pixel of the heightmap. So with no interpolation, I only get 256 levels of precision as that's what 8-bits give you.

So, I'd like to use 16-bit per-channel precision for my heightmaps. I've read on how to do a Custom Importer, but I'm basically stuck at the part where I need to make an Image instance based on the png file.

I have no idea where to go from here, or whether it's even feasable for a noob like me :disappointed:

Thanks in advance.

arakash

So, I've decided to try and read PNG from the raw bytes. This is new to me, but reading the PNG Specification has lead me to be able to read the width and height of the image file, as well as the bit depth and color type.

Reading the pixel samples seem beyond me, however.

From the PNG specification, it says:

Pixels are always packed into scanlines with no wasted bits between pixels. Pixels smaller than a byte never cross byte boundaries; they are packed into bytes with the leftmost pixel in the high-order bits of a byte, the rightmost in the low-order bits. Permitted bit depths and pixel types are restricted so that in all cases the packing is simple and efficient.

Which I have no idea what that means :confounded:

Anyone? :lol:

Calinou

Godot always imports PNGs as 8 bpc per design to save memory. The reason for doing this is that many users are unaware that their PNGs are saved with higher bit depth, as some software out there will use > 8 bpc by default. If Godot imported the PNG with its original bit depth, the engine would end up wasting memory (and increasing projects' installed file size).

To import higher precision images, you can use the OpenEXR (.exr) or Radiance HDR (.hdr) formats instead of PNG. The downside is that these formats can't be loaded in an exported project without being imported in the editor beforehand, as their relevant modules are only compiled in editor builds.

I suppose an opt-in import option could be added to import PNGs with their original bit depth, but this should be discussed in a proposal.

Edit: Discussion opened: https://github.com/godotengine/godot-proposals/discussions/4143

arakash

@Calinou said: To import higher precision images, you can use the OpenEXR (.exr) or Radiance HDR (.hdr) formats instead of PNG. The downside is that these formats can't be loaded in an exported project without being imported in the editor beforehand, as their relevant modules are only compiled in editor builds.

That would be a problem. the heightmaps need to be loaded from disk when running the game. I suppose I can save it in a custom float[] array format for the exported version maybe?

@Calinou said: I suppose an opt-in import option could be added to import PNGs with their original bit depth, but this should be discussed in a proposal.

Right. I think having the support is a good idea as it's very common for heightmap based terrains.

arakash

Little update in case someone else is interested in this:

Support for 16 bit png was propsed and rejected on the grounds that it wasn't useful enough and that it might make some games use more hard disk space without really needing it (which is true).

I finally found a program than can batch convert from PNG to EXR (Affinity Photo) and it seems to work great. I can even save the modified (after sculpting terrain) using image.save_exr() which is great.

However, I was hoping to use 16-bit integer precision - but it looks like that's not supported in godot. There's 16 bit float and 32-bit float - but that might lead to accumulative errors after loading/saving terrain many times - which is especially worrisome with regards to chunk seams.

I'm not sure how big of an issue this might be though, and I might be able to reduce these errors by clamping the RGB values - but of course, that takes extra processing time which is a bit annoying.

Calinou

However, I was hoping to use 16-bit integer precision - but it looks like that's not supported in godot. There's 16 bit float and 32-bit float - but that might lead to accumulative errors after loading/saving terrain many times - which is especially worrisome with regards to chunk seams.

I don't see how using floating-point formats would lead to error accumulation, as long as your project stores the heightmap coordinates using floats. Error accumulation would only occur if you convert the values to integers aftere loading them.

arakash

@Calinou said:

However, I was hoping to use 16-bit integer precision - but it looks like that's not supported in godot. There's 16 bit float and 32-bit float - but that might lead to accumulative errors after loading/saving terrain many times - which is especially worrisome with regards to chunk seams.

I don't see how using floating-point formats would lead to error accumulation, as long as your project stores the heightmap coordinates using floats. Error accumulation would only occur if you convert the values to integers aftere loading them.

Oh.... heh :lol: that's good to know. I read that someone had that issue - I guess they were converting them to integers then - I don't need that.

Also, after testing, it seems to work pretty well!

arakash

Another update:

So, EXR Works great. Except for when your other software doesn't support it and you have to do file conversion. In my case, TerreSculptor (terrain generation software) doesn't support EXR so I have to batch convert thousands of files.

Of course, tools exist to do this and it would be fine for a few files here and there. But in my case, dealing with 16384 files, many tools will outright fail due to bad program design and some will be very slow because it doesn't utilize multiple cores.

Affinity Photo does work, but the UI hangs and sometimes crashes, so it's not suitable. The popular command line ImageMagick tool only has EXR support when compiled from source with extra OpenEXR libraries, something that many will struggle to do and is cumbersome (I have yet to attempt this).

I found a script for blender that essentially batch converts any image in a source directory and saves to EXR (since blender has EXR support built in). But the script is for blender 2.79, but fine, I tried it and it basically doesn't work. The resulting images are all gray somehow.

So, I'm basically stuck at this point until Godot gets support for 16 bit PNG (the author of TerreSculptor said that it will be unlikely for him to add exr support in the near future, due to lack of funding for the project).