Dynamically generating textures at runtime from C++

Wallerton

I have some C++ code that procedurally generates image data that varies over time. I want to use this data to set a texture within Godot every frame, so that the generated images play like a video. Are there any examples of setting textures based on raw bytes as opposed to a file, particularly from GDNative C++?

Also, how can I debug GDNative C++ code, or at least get the error messages for it? When there is a problem with my GDNative resource, godot just tells me socket error 104.

ben-gachowski

There is the godot::Image class in the generated header file Image.hpp It has multiple supported internal formats and notably has the functions:

void create_from_data(const int64_t width, const int64_t height, const bool use_mipmaps, const int64_t format, const PoolByteArray data); Error load_jpg_from_buffer(const PoolByteArray buffer); Error load_png_from_buffer(const PoolByteArray buffer); Error load_webp_from_buffer(const PoolByteArray buffer); PoolByteArray get_data() const;

Which look like the functions you'd be interested in. If you create a Texture, you can call Ref<Image> get_data() const to get it's internal Image, if it has one.

Wallerton

Thanks for the reply. This is what I've found that basically works. In process: Image im = Image::new(); ImageTexture imTex = ImageTexture::_new(); im->create(width, height, false, Image::FORMAT_RGBA8); im->lock(); for(int x=0; x<width; x++){ for(int y=0; y < height; y++){ // call im->set_pixel to create the image } } im->unlock(); imTex->set_storage(ImageTexture::STORAGE_RAW); imTex->create_from_image(Ref<Image>(im)); set_texture(Ref<ImageTexture>(imTex));

It's not totally ideal because, first, I'm calling set_pixel a lot of times instead of loading the image directly from a byte array, and second, I'm having to create a new image and new texture every frame instead of changing an existing one, and that ought to have some overhead. The program crashes if I try to lock an Image that was already used to create an ImageTexture. It also crashes if I try to call set_data on an ImageTexture more than once.

Fortunately I am still getting a decent framerate (with some flickering, though...), and there doesn't appear to be a memory leak.

ben-gachowski

If you create the image texture with the flag Texture::Flags::FLAG_VIDEO_SURFACE you should be allowed to write your image data into a PoolVector<uint8_t> and then create a new Image from the data, and call set_data on the Texture, without having to recreate the texture every frame. I think you have to recreate the Image every frame however. Additionally you can probably get significantly better performance by writing the pixel data row by row, instead of column by column, by changing the order from x then y, to y then x:

for(int y=0; y<height; y++){
	for(int x=0; x < height; x++){
		// write one pixel to the PoolVector<uint8_t>
		// Since the type is RGBA8, you need to write 4 bytes for each pixel.
	}
}

This is pretty much how they implemented video streaming to a texture in the main engine, so it should be nearly as performant as that. For more info on how to do it right you can look at the streaming code in the video_stream_webm.cpp file.