estimateShape - estimates the shape of the attended proto-object region. shapeData = estimateShape(salmap,saliencyData,winner,saliencyParams) Estimates the shape of the attended proto-object region from the saliencyData: salmap: the saliency map as returned by makeSaliencyMap. saliencyData: the saliencyData as returned by makeSaliencyMap. winner: the winning location in saliency map coordinates. saliencyParams: the necesary parameters. shapeData: structure containing information about the shape of the attended regions, with the following fields: origImage: the Image structure for the source image. winner: the winning location in saliency map coordinates. winningMap: the map for the most salient feature at the winner location. iorMask: the mask used for shape-based inhibition of return. binaryMap: a binary map of the attended region. segmentedMap: the winning map segmented by the binary map. shapeMap: a smoothed version of segmentedMap. date: the time and date of the creation of this structure. If finding an appropriate map for segmentation failed, an empty shapeData structure is returned. The possible params.shapeModes for shape estimation are: 'None': no shape processing. 'shapeSM': use the saliency map. 'shapeCM': use the conspicuity map with the largest contribution to the saliency map at the attended location. 'shapeFM': use the feature map with the largest contribution to that conspicuity map. 'shapePyr': use the pyramid level (center or surround level) with the largest ontribution to that feature map. For details of this method see: Walther, D., and Koch, C. (2006). Modeling attention to salient proto-objects. Neural Networks 19, pp. 1395-1407. See also makeSaliencyMap, evolveWTA, dataStructures, runSaliency, applyIOR, shapeIOR, defaultSaliencyParams.
0001 % estimateShape - estimates the shape of the attended proto-object region. 0002 % 0003 % shapeData = estimateShape(salmap,saliencyData,winner,saliencyParams) 0004 % Estimates the shape of the attended proto-object region from the saliencyData: 0005 % 0006 % salmap: the saliency map as returned by makeSaliencyMap. 0007 % saliencyData: the saliencyData as returned by makeSaliencyMap. 0008 % winner: the winning location in saliency map coordinates. 0009 % saliencyParams: the necesary parameters. 0010 % 0011 % shapeData: structure containing information about the shape of 0012 % the attended regions, with the following fields: 0013 % origImage: the Image structure for the source image. 0014 % winner: the winning location in saliency map coordinates. 0015 % winningMap: the map for the most salient feature at the winner location. 0016 % iorMask: the mask used for shape-based inhibition of return. 0017 % binaryMap: a binary map of the attended region. 0018 % segmentedMap: the winning map segmented by the binary map. 0019 % shapeMap: a smoothed version of segmentedMap. 0020 % date: the time and date of the creation of this structure. 0021 % 0022 % If finding an appropriate map for segmentation failed, an empty 0023 % shapeData structure is returned. 0024 % 0025 % The possible params.shapeModes for shape estimation are: 0026 % 'None': no shape processing. 0027 % 'shapeSM': use the saliency map. 0028 % 'shapeCM': use the conspicuity map with the largest contribution 0029 % to the saliency map at the attended location. 0030 % 'shapeFM': use the feature map with the largest contribution 0031 % to that conspicuity map. 0032 % 'shapePyr': use the pyramid level (center or surround level) with 0033 % the largest ontribution to that feature map. 0034 % 0035 % For details of this method see: 0036 % Walther, D., and Koch, C. (2006). Modeling attention to salient 0037 % proto-objects. Neural Networks 19, pp. 1395-1407. 0038 % 0039 % See also makeSaliencyMap, evolveWTA, dataStructures, runSaliency, 0040 % applyIOR, shapeIOR, defaultSaliencyParams. 0041 0042 % This file is part of the SaliencyToolbox - Copyright (C) 2006-2008 0043 % by Dirk B. Walther and the California Institute of Technology. 0044 % See the enclosed LICENSE.TXT document for the license agreement. 0045 % More information about this project is available at: 0046 % http://www.saliencytoolbox.net 0047 0048 function shapeData = estimateShape(salmap,saliencyData,winner,params) 0049 0050 if strcmpi(params.shapeMode,'None') 0051 shapeData = []; 0052 return 0053 end 0054 0055 shapeData.origImage = salmap.origImage; 0056 shapeData.winner = winner; 0057 0058 tmp1 = cat(1,saliencyData.CM); 0059 tmp2 = cat(3,tmp1.data); 0060 [mx,CMidx] = max(tmp2(winner(1),winner(2),:)); 0061 0062 % first default: winning map is saliency map 0063 winMap = salmap; 0064 winPos = {winner}; 0065 0066 % need to go deeper into the maps? 0067 if ((mx > 0) & ~strcmp(params.shapeMode,'shapeSM')) 0068 0069 % find the biggest contributing conspicuity map 0070 CMidx = CMidx(randInt(length(CMidx))); 0071 tmp1 = cat(1,saliencyData(CMidx).FM(:)); 0072 tmp2 = cat(3,tmp1.data); 0073 [mx,FMidx] = max(tmp2(winner(1),winner(2),:)); 0074 0075 % found our winning conspicuity map 0076 winMap(end+1) = saliencyData(CMidx).CM; 0077 winPos{end+1} = winner; 0078 0079 % need to go deeper? 0080 if ((mx > 0) & ~strcmp(params.shapeMode,'shapeCM')) 0081 0082 % our next bet are the feature maps that contribute 0083 % to the wining conspicuity map 0084 FMidx = FMidx(randInt(length(FMidx))); 0085 winMap(end+1) = saliencyData(CMidx).FM(FMidx); 0086 winPos{end+1} = winner; 0087 0088 % need to go deeper still? 0089 if (strcmp(params.shapeMode,'shapePyr')) 0090 0091 % now we compare the contributing center and surround maps 0092 cen = saliencyData(CMidx).csLevels(FMidx).centerLevel; 0093 sur = saliencyData(CMidx).csLevels(FMidx).surroundLevel; 0094 [pyrIdx,tmp] = ind2sub(size(saliencyData(CMidx).FM),FMidx); 0095 cenMap = saliencyData(CMidx).pyr(pyrIdx).levels(cen); 0096 surMap = saliencyData(CMidx).pyr(pyrIdx).levels(sur); 0097 0098 % extract the values of the cen and sur maps at the winner location 0099 wCM = size(saliencyData(CMidx).CM.data,2); 0100 cenWin = round(winner/wCM * size(cenMap.data,2)); 0101 surWin = round(winner/wCM * size(surMap.data,2)); 0102 cenWin = max(min(cenWin,size(cenMap.data)),[1 1]); 0103 surWin = max(min(surWin,size(surMap.data)),[1 1]); 0104 cenVal = cenMap.data(cenWin(1),cenWin(2)); 0105 surVal = surMap.data(surWin(1),surWin(2)); 0106 0107 % compare center and surround and store the winner 0108 if (abs(cenVal) > abs(surVal) & (min(size(cenMap.data)) > 7)) 0109 winMap(end+1) = cenMap; 0110 winVal = cenVal; 0111 winPos{end+1} = cenWin; 0112 else 0113 winMap(end+1) = surMap; 0114 winVal = surVal; 0115 winPos{end+1} = surWin; 0116 end 0117 0118 % renormalize the cen or sur map values for better segmentation 0119 winMap(end).data = (1 - abs(winMap(end).data - winVal) / winVal).^2; 0120 end 0121 end 0122 end 0123 0124 % now we have extracted all the maps we need 0125 debugMsg(winMap(end).label) 0126 debugMsg(sprintf('Value at winning location: %g',... 0127 winMap(end).data(winPos{end}(1),winPos{end}(2)))); 0128 gotMap = 0; 0129 0130 % let's see who behaves nicely for segmentation 0131 for idx = length(winMap):-1:1 0132 switch params.segmentComputeType 0133 case 'Fast' 0134 binMap = fastSegmentMap(winMap(idx),winPos{idx}); 0135 case 'LTU' 0136 binMap = LTUsegmentMap(winMap(idx),winPos{idx}); 0137 otherwise 0138 error(['Unknown segmentComputeType: ' params.segmentComputeType]); 0139 end 0140 0141 % check that we actually segmented something, but not too big (< 10%) 0142 areaRatio = sum(binMap.data(:)) / prod(size(binMap.data)); 0143 if ((areaRatio > 0) & (areaRatio < 0.1)) 0144 0145 % this guy looks good - let's keep him! 0146 shapeData.winningMap = winMap(idx); 0147 shapeData.winner = winPos{idx}; 0148 0149 % for the IOR mask, we don't want to smooth the shape 0150 shapeData.iorMask = binMap; 0151 shapeData.iorMask.data = imdilate(shapeData.iorMask.data,strel('disk',2)); 0152 shapeData.iorMask.label = 'IOR mask'; 0153 0154 % for the binary map, erode the shape a bit 0155 binMap.label = 'binary shape map'; 0156 se = [[0 0 1 0 0];[0 1 1 1 0];[1 1 1 1 1];[0 1 1 1 0];[0 0 1 0 0]]; 0157 tmp = imclose(imopen(binMap.data,se),se); 0158 newMap = []; 0159 if (tmp(winPos{idx}(1),winPos{idx}(2)) > 0) 0160 if (sum(tmp(:)) > 0) 0161 newMap = tmp; 0162 end 0163 else 0164 se = [[0 1 0];[1 1 1];[0 1 0]]; 0165 tmp = imclose(imopen(binMap.data,se),se); 0166 if ((tmp(winPos{idx}(1),winPos{idx}(2)) > 0) && (sum(tmp(:)) > 0)) 0167 newMap = tmp; 0168 end 0169 end 0170 if ~isempty(newMap) 0171 lab = bwlabel(newMap,4); 0172 binMap.data = double(lab == lab(winPos{idx}(1),winPos{idx}(2))); 0173 end 0174 shapeData.binaryMap = binMap; 0175 gotMap = 1; 0176 break; 0177 end 0178 end 0179 0180 % huh - no success in segmentation? Just return empty then 0181 if (~gotMap) 0182 shapeData = []; 0183 return; 0184 end 0185 0186 % Hurray, we have a nicely segmented map - let's compute a few more things 0187 0188 % The segmented map is just winning map * binary map 0189 shapeData.segmentedMap.origImage = shapeData.winningMap.origImage; 0190 shapeData.segmentedMap.label = 'segmented shape map'; 0191 shapeData.segmentedMap.data = shapeData.winningMap.data .* shapeData.binaryMap.data; 0192 shapeData.segmentedMap.date = timeString; 0193 shapeData.segmentedMap.parameters = shapeData.winningMap.parameters; 0194 0195 % The shape map is a smoothed version of the binary map 0196 shapeData.shapeMap.origImage = shapeData.winningMap.origImage; 0197 shapeData.shapeMap.label = [shapeData.binaryMap.label ' - rescaled']; 0198 tmp = imresize(shapeData.binaryMap.data,... 0199 shapeData.shapeMap.origImage.size(1:2),'nearest'); 0200 kernel = gaussian(0,15,15); 0201 tmp = normalizeImage(sepConv2PreserveEnergy(kernel,kernel,tmp),[0,3]); 0202 shapeData.shapeMap.data = clamp(tmp,0,1); 0203 0204 shapeData.shapeMap.date = timeString; 0205 shapeData.shapeMap.parameters = shapeData.winningMap.parameters; 0206 0207 shapeData.date = timeString;