The issue is probably from the texture linear interpolation, though its called filter
in the Godot import settings. If you turn this off, then pixels on the edges of each frame shouldn't leak into the next, though the texture itself won't be as filtered.
Another solution is to use a texture region that is slightly smaller than each grid cell in the texture. Especially if each texture doesn't take up the entire grid cell, this solution should work.