Although interacting with objects appears to compress spatial memory, the full extent of this effect is unknown. To address this issue, I examined whether action-based compression could extend to surrounding objects or environments. Participants examined objects in two nested regions and recalled object-pair distances for each region. In one region, participants inspected objects manually or visually. In the other region, they only inspected objects visually. By having participants interact with a subset of objects, I assessed whether interaction compresses memory for other objects. By marking the boundary between regions, I also assessed whether interaction compresses memory for other environments. When the boundary was unmarked, manual interaction led participants to recall shorter distances throughout the environment. However, marking the boundary led participants to recall shorter distances overall. While these findings suggest that manual interaction can compress spatial memory throughout an environment, the effects of interaction across environments remain inconclusive.