I think I figured it out. I have 2 Area2Ds, one attached to the player and one attached to the box. Both Area2Ds are made so that they are slightly bigger than the object, except on the top of the box and at the bottom of the feet of the player, where I do want the objects to collide with each other. When the Area2Ds detect the other object, they send a signal to the parent object, then in code I disabled the collision mask that would make them collide with each other. Then I enable them again when the body exits the Area2D.
This way I don't have to worry about the objects falling through the floor if I were to disable their colliders or mess with their physics. I will still have to figure out how to move the box, but I'll probably just move it manually via code as long as the player is within it's Area2D (I changed the box to a Kinematic Body).
Thank you for your help.