OpenCV - iterate over each blob in a binary image and use it as mask

Question

I have a binary image and a color image of the same size. I need to iterate each blob (white pixel blocks) of the binary image and use it as a mask and find the mean color of this blob region from the color image.

I have tried:

HierarchyIndex[] hierarchy;
Point[][] contours;
binaryImage.FindContours(out contours, out hierarchy, RetrievalModes.List, ContourApproximationModes.ApproxNone);

    using (Mat mask = Mat.Zeros(matColor.Size(), MatType.CV_8UC1))
        foreach (var bl in contours)
            if (Cv2.ContourArea(bl) > 5)
            {
                mask.DrawContour(bl, Scalar.White, -1);                                                    
                Rect rect = Cv2.BoundingRect(bl);
                Scalar mean = Cv2.Mean(colorImage[rect], mask[rect]);

                mask.DrawContour(bl, Scalar.Black, -1);
            }

which works for the blobs not having holes. However in my case I have many blob regions having huge holes that affects the mean calculation.

I couldn't figure it out how to solve it using the hierarchy info; or with another approach.

(My code is for OpenCVSharp but answer in any other wrapper or language is wellcome.)

Edit: I've added an example image. The traffic signs part is the problem.

Actually I think I have solved this problem with this method:

using PLine = List<Point>;
using Shape = List<List<Point>>;
internal static IEnumerable<Tuple<PLine, Shape>> FindContoursWithHoles(this Mat mat)
        {
            Point[][] contours;
            HierarchyIndex[] hierarchy;
            mat.FindContours(out contours, out hierarchy, RetrievalModes.Tree, ContourApproximationModes.ApproxNone);

            Dictionary<int, bool> dic = new Dictionary<int, bool>();
            for (int i = 0; i < contours.Length; i++)
                if (hierarchy[i].Parent < 0)
                    dic[i] = true;

            bool ok = false;
            while (!ok)
            {
                ok = true;
                for (int i = 0; i < contours.Length; i++)
                    if (dic.ContainsKey(i))
                    {
                        bool isParent = dic[i];
                        var hi = hierarchy[i];
                        if (hi.Parent >= 0) dic[hi.Parent] = (!isParent);
                        if (hi.Child >= 0) dic[hi.Child] = (!isParent);
                        while (hi.Next >= 0)
                        {
                            dic[hi.Next] = isParent;
                            hi = hierarchy[hi.Next];
                            if (hi.Parent >= 0) dic[hi.Parent] = (!isParent);
                            if (hi.Child >= 0) dic[hi.Child] = (!isParent);
                        }
                        hi = hierarchy[i];
                        while (hi.Previous >= 0)
                        {
                            dic[hi.Previous] = isParent;
                            hi = hierarchy[hi.Previous];
                            if (hi.Parent >= 0) dic[hi.Parent] = (!isParent);
                            if (hi.Child >= 0) dic[hi.Child] = (!isParent);
                        }
                    }
                    else
                        ok = false;
            }
            foreach (int i in dic.Keys.Where(a => dic[a]))
            {
                PLine pl = contours[i].ToList();
                Shape childs = new Shape();
                var hiParent = hierarchy[i];
                if (hiParent.Child >= 0)
                {
                    childs.Add(contours[hiParent.Child].ToList());
                    var hi = hierarchy[hiParent.Child];
                    while (hi.Next >= 0)
                    {
                        childs.Add(contours[hi.Next].ToList());
                        hi = hierarchy[hi.Next];
                    }
                    hi = hierarchy[hiParent.Child];
                    while (hi.Previous >= 0)
                    {
                        childs.Add(contours[hi.Previous].ToList());
                        hi = hierarchy[hi.Previous];
                    }
                }

                yield return Tuple.Create(pl, childs);
            }
        }

By drawing the holes as black, we can use each blob as a single mask:

var blobContours = blobs.FindContoursWithHoles().ToList();
using (Mat mask = Mat.Zeros(mat0.Size(), MatType.CV_8UC1))
    for (int i = 0; i < blobContours.Count; i++)
    {
        var tu = blobContours[i];
        var bl = tu.Item1;
        if (Cv2.ContourArea(bl) > 100)
        {
            mask.DrawContour(bl, Scalar.White, -1);
            foreach (var child in tu.Item2)
                mask.DrawContour(child, Scalar.Black, -1);
            Rect rect = Cv2.BoundingRect(bl);
            Scalar mean = Cv2.Mean(mat0[rect], mask[rect]);
        }
    }

I think there should be an easier way.

And yet there is another problem. In some cases, an individual red part of the sign (which is a seperate white blob) does not found as a parent outside circle and a child inside circle, but a large parent contour outside with two circles as children (ie. hole inside another hole, makes a seperate blob which is not found as a parent). Yes it is hierarchically correct but does not help me. I hope I could make my self clear, sorry for my English.

Answer 1

@Miki thank you very much. I was able to achieve what I want using ConnectedComponents. Its simple and fast:

var cc = Cv2.ConnectedComponentsEx(binaryImage, PixelConnectivity.Connectivity8);
foreach (var bl in cc.Blobs)
    using (Mat mask = new Mat())
    {
        cc.FilterByBlob(binaryImage, mask, bl);
        Rect rect = bl.Rect;
        Scalar mean = Cv2.Mean(colorImage[rect], mask[rect]);
    }