Generate Perspective Transformation Matrix?

gauntlet

I am trying to generate the transformation matrix necessary to change the perspective of a rectangular image. I can't find if there are any functions in Godotscript that would help with this?

I am able to apply a transformation matrix but I can't figure out how to get the values for the 3x3 matrix from the two sets of coordinate points.

I can go into an image editing program like GIMP and get the matrix values and then apply them in Godot, but I want to be able to define the coordinates in Godot and do it automatically.

Does this make sense at all?

Thanks

cybereality

The 3x3 matrix is called Basis: https://docs.godotengine.org/en/stable/classes/class_basis.html

You can create one using either Euler angles or a Quat, which will give you just rotation. You can use the scaled method to scale it. Then create a Transform ( https://docs.godotengine.org/en/stable/classes/class_transform.html ) with the Basis you just created and the origin parameter set to the position/translation you want.

gauntlet

Well the problem is that I don't actually know what rotation, scaling, etc. It needs. I just have reference points.

gauntlet

I'm actually not sure that I need 3x3. The 2x3 Transform2D might work but I don't know how to get it from two sets of coordinates. I guess I'll try just using Vectors that are one set subtracted from the other.

Thanks,

xyz

It's a 3x3 matrix but calculating it is not trivial. And I'm not even sure if such matrix could be used directly for distorting images inside Godot. If you can get away with just using a textured quad and moving its vertices to wanted positions - I'd suggest using that. It's not "real" perspective distortion though.

To get a proper 3x3 matrix, you need to solve a system of 8 equations with 8 unknowns (last element of matrix is 1). The approach is described here but it uses some numeric library to actually solve the equations. Not sure if you can automate this with Godot's math facilities.

Some alternative ways of looking at it can be found here, but it all boils down to same thing.

gauntlet

Thank you. The matrix can definitely be used once you have it. I've already done it using matrix values calculated from gimp. I just scan through pixels and back calculate where they should source color from.

You're right that finding the matrix values is a lot harder than I expected.

xyz

Oh, you're manually applying it to pixels? Sure, it can be used that way...

You could "hardcode" a solution to this particular set of linear equations. WolframAlpha can probably solve it for you and print out solutions using symbolic values. Just copy them into your code.

cybereality

Can you explain what these 2 sets of coordinates represent?

xyz

I don't know what was meant by "2 sets of coordinates" but such matrix is calculated from 4 distorted corner positions, that's 8 values in total. The first link I posted describes it quite well. It's not that complicated although matrix notation can look intimidating at first :)

gauntlet

@xyz Yeah, I'm manually applying to pixels but I want to be able to generate the matrix from the 4 distorted corner points (I called them sets of coordinates) that you mention.

I'm thinking I might get by with a 2x3 matrix because the last row should be barely different from [0, 0, 1] so maybe I can just assume those values.

cybereality

So you can calculate a 3x3 matrix from points, you would only need 2 vectors, but they would need to be in 3D. So if you take two adjacent edges (pointing away from the same corner) do a cross product, now you have 3 Vector3s which define a coordinate space. Then create a Basis with the 3 vectors as parameters (this defines the x/y/z axes for the space). This will produce a 3x3 matrix with the proper rotation/orientation.

xyz

@cybereality This is not-affine transformation matrix. In that regard it's similar to "regular" camera perspective projection matrix which too is non-linear. You can't build such matrix from basis vectors.

xyz

@gauntlet said: I'm thinking I might get by with a 2x3 matrix because the last row should be barely different from [0, 0, 1] so maybe I can just assume those values.

There's still 6 values to calculate from the system of 6 equations. No way around it.

xyz

Ok so here's the system of equations that needs to be solved

x0h0 + y0h1 + h2 - u0x0h6 - u0y0h7 = u0 x0h3 + y0h4 + h5 - u0x0h6 - u0y0h7 = v0

x1h0 + y1h1 + h2 - u1x1h6 - u1y1h7 = u1 x1h3 + y1h4 + h5 - u1x1h6 - u1y1h7 = v1

x2h0 + y2h1 + h2 - u2x2h6 - u2y2h7 = u2 x2h3 + y2h4 + h5 - u2x2h6 - u2y2h7 = v2

x3h0 + y3h1 + h2 - u3x3h6 - u3y3h7 = u3 x3h3 + y3h4 + h5 - u3x3h6 - u3y3h7 = v3

where: x, y are coordinates of original points u, v are coordinates of distorted points h0-h7 are variables you need to solve for, these then go directly into the matrix

If you have Mathematica or Pro account at WolframAlpha, you can just paste:

solve x0h0+y0h1+h2-u0x0h6-u0y0h7=u0, x0h3+y0h4+h5-u0x0h6-u0y0h7=v0, x1h0+y1h1+h2-u1x1h6-u1y1h7=u1, x1h3+y1h4+h5-u1x1h6-u1y1h7=v1, x2h0+y2h1+h2-u2x2h6-u2y2h7=u2, x2h3+y2h4+h5-u2x2h6-u2y2h7=v2, x3h0+y3h1+h2-u3x3h6-u3y3h7=u3, x3h3+y3h4+h5-u3x3h6-u3y3h7=v3 for h0,h1,h2,h3,h4,h5,h6,h7

gauntlet

So if I use Transform2d(), the matrix it creates is something different?

xyz

@gauntlet said: So if I use Transform2d(), the matrix it creates is something different?

What did you use with values from gimp?

Looking at the docs for both Transform and Transform2d it seems their apis are geared completely towards affine transformations. And I don't see a way to directly set values in their matrices. So this kind of matrix seemingly can't be built using these types. But I might be wrong...

cybereality

Oh yeah, you're right. I was trying to code it myself, but I found this which looks interesting. https://cenullum.itch.io/skew-stretching-godot-engine-editor-tool

cybereality

Can you describe what you are trying to accomplish or maybe provide an image? There may be an easier way to do it, I can look into it.

xyz

@cybereality said: Oh yeah, you're right. I was trying to code it myself, but I found this which looks interesting. https://cenullum.itch.io/skew-stretching-godot-engine-editor-tool

By the looks of it it's just bilinear interpolation, not true perspective distort. It's analogous to mapping a texture to a 2d quad and moving the vertices. But as I already said, if this looks good enough for what you need - use it. It's far less complicated than true perspective. And somebody already made the script :)

And there's always third option - use 3d sprite and actual 3d camera. It'll do all the perspective for you :)

gauntlet

@xyz said:

What did you use with values from gimp?

I used the basic formula for applying the transformation to coordinates:

x' = (xh0 + yh1 + h2) / (xh6 + xh7 + 1) y' = (xh3 + yh4 + h5) / (xh6 + xh7 + 1)

I don't actually need to set the values in the matrices, I just need to be able to extract the values since the application is just the simple equations above.

@cybereality said: Can you describe what you are trying to accomplish or maybe provide an image? There may be an easier way to do it, I can look into it.

I have a webcam looking down on a table with various objects on it. The camera can't always be directly over the center of the table so the image is skewed/rotated/translated. I want to undistort this image to be used in the rest of my Godot program. I also need to access the pixels of the image after the transform, it's not just aesthetic. Here is a basic sketch:

Using the coordinates of the corners of those images, it is possible to generate the matrix values needed to perform the transform, I'm just looking for the simplest way to do that in Godot, since a one-time solution from gimp is not useful.

I'll check out the link once I get home and make sure that Transform2d() isn't what I need.