Segment Vehicles in crowded scene.

Hi, I have a video taken on the traffic light scene. My task is to segment every single car/vehicles when the red light of the traffic light is activated, which mean I have to segment all the vehicles when they are in queue up position. I don't have problem to segment every single vehicles/car when they are moving since I can use either optical flow or frame differences technique.

I did upload the video in youtube. Here is the link

<http://www.youtube.com/watch?v=u785JkXWCBA&feature=youtu.be>.

According to this video, at time 00:43 the vehicles are start to Que-up that need my attention to segment it.

Anybody that know the method or opinion on how to perform this are most welcome. Thanks

p/s : 1) I don't have background of this video to perform background subtraction technique.

Answer 1

[David Young]

I suggest that you approach this by trying to generate a background image from the video. You are able to segment the images when the traffic is moving, so you can reliably find regions that belong to the background in any given frame. You can combine these to get an overall background image. (You would want to update this dynamically in a real system.) Once you have a background image, you can use background subtraction as you point out.

It may be that you can get an adequate background image by simply averaging a large number of frames when the traffic is moving. This might be worth a try first.

The issue in the traffic monitoring problem is usually time scale. If the lifetime of a queue of vehicles is large compared to the timescale of illumination changes, you need to work harder to produce a reliable background image. Normalising to deal with illumination changes can help. You also need to address question like what happens if a vehicle parks in the scene - when should it be treated as background? However, these aren't problems if all you need to do is analyse the video clip you showed.

[Image Analyst]

I think you mean the "mode" rather than the average, don't you? I think the mode would have a better shot at picking out the true background value than the mean, which is influenced by whether vehicles are in there, particularly for shorter sequences where a high percentage of the video is taken up by a variety of vehicles.

[David Young]

You're right, the mode would perform better - though it needs more resources to compute, I think.

[fariz]

Hi David,

Thanks a lot for your input David. Appreciate that, by the way do u have any example or paper that i can refer to in order to use your suggestion. Thanks..

[David Young]

Not realy, sorry. A search of the computer vision literature should reveal something - probably from quite some time ago - that describes similar techniques. (We used this kind of method for the work described in L. Xu, D. Young, and D. C. Hogg, Building a Model of a Road Junction Using Moving Vehicle Information, British Machine Vision Conference, Leeds, 1992, but it may not go into much detail.) Are you asking because you need more details in order to write an implementation, or do you want a reference to cite in a publication?

[fariz]

Hi David,

Yes, I certainly need details on your approach above to try on my video as I'm still in learning curve in video processing. Appreciate if u can provide me some example.

Answer 2

[Image Analyst]

You might try Gaussian Mixture Models, such as is described here http://www.ai.mit.edu/projects/vsam/Publications/stauffer_cvpr98_track.pdf GMM is used quite commonly for background estimation.

Also check out moving object removal as illustrated on this page: http://www.mee.tcd.ie/~sigmedia/Research/RigRemoval

[fariz]

These are helping. Thanks a lots guys.