Adding marks to images to aid detection and segmentation by mask-rcnn? or feed additional input somehow?

Question

Hi all sorry for coming in only when I need help again. Also I'm not sure if this is the right forum for this question. It's not exactly about coding, more about model application.

I'm looking to use something like Mask-RCNN (also looking into Hybrid Task Cascade) for both object detection and instance segmentation.

The issue is that along with a dataset of images, I also have additional parameters I want to incorporate to aid the Mask-RCNN with detection and segmentation.

Say, assuming I have a dataset of buildings and I want to detect windows, doors, balconies, etcs. And for each image I have additional parameters like architectural style, year-built, usage (shop, office or residential), or whatever else I may come up with.

How do I feed those additional information to the Mask-RCNN model?

I'm thinking of doing something like modifying the corner pixels of the images, like red on the top-left corner for office, blue for shop, green for residence, and different shades of grey on the top-right corner for year built, for example. Would that work? Would it aid the model in detection and segmentation? Is there any other way of feeding additional information to the model?

Sorry the question is a bit long, but I bolded the main question points to aid with reading. Thanks for reading and for any confirmation or suggestion!

question from:https://stackoverflow.com/questions/65660834/adding-marks-to-images-to-aid-detection-and-segmentation-by-mask-rcnn-or-feed-a

Answer 1

Waitting for answers