A better way to do it would be to maintain canvas Path2d objects to define the enclosed regions, which can be instantly filled with context.fill()

No need for complex pixel manipulation stuff

This also lets you do things like let the user move things around.