Webcam Capture in Matlab

The Image Acquisition Toolbox in Matlab (Windows version) allows one to interface Matlab with a Webcam. This is available from R2007a (not sure about earlier versions). Similar to the audio recording object created earlier, here we create a videoinput object. But before that is done, Matlab needs to find out what are the webcam devices that are connected to your computer.

Firstly, a imaqhwinfo gives information about the existing adaptors for your webcam device. You can get more information on each adapter, by using imaqhwinfo('winvideo') where winvideo is one of the adaptors. In this, (if you have a device connected) you shall get a Device IDs attached to your webcam device. Further information pertaining to the device can be obtained by imaqhwinfo('winvideo',1) where 1 is the Device ID you saw earlier.

This gives you much needed information regarding the capture device. The resolution (800×600, 1024×768, 1600×1200, etc.), format (RGB, YUV, etc.) which needs to be selected when creating a video object.

Armed with all this imaqhwinfo (image acquisition hardware information) you are ready to create your own video object.

vidobj = videoinput('winvideo',1,'RGB_1024x768');

‘RGB_1024x768′ was just the format that I selected. You should use one of those that were available in your device info query. The most important command now would be to start your video object start(vidobj). It is at this point, or during the creation of video object, that the light (if any) on your webcam would start glowing indicating capture.

You can obtain snapshots of capture by using the frame = getsnapshot(vidobj); or view the continuous stream of frames by saying preview(vidobj);.

A safe closure (unlocking of the video handles) of the video object is extremely important so that it can be started again easily. A stop(vidobj) followed by delete(vidobj) is the best way to follow.

Another point to note is that all external capture devices, are locked by software which try to access it. Thus, you would get errors like Device not ready, or Device already in use in case you are already viewing the capture stream in any other software. So its recommended that you cleanly stop that software first and then let Matlab take over.

There are a variety of options that are not discussed here for lack of purpose like an automatic trigger (after a defined interval). All the options can be seen by imaqhelp(videoinput).

You now have the power of both Audio and Image / Video Capture now with which amazing tricks can be played ;)

UPDATE (2013-Aug-23): Might be interesting to just start exploring using imaqtool.

About these ads

49 thoughts on “Webcam Capture in Matlab

  1. P

    Hi,
    Thank you for your help. Would you please let me know how I can save the video? with preview I can just see the video from the webcam but I do not know how to save it!! Thanks

    Reply
    1. Makarand Tapaswi Post author

      Hi
      You could think of using getsnapshot followed by imwrite in a while loop until some condition. This might not be the most efficient system, but should work quite fine. The “fps” it provides may or may not be enough depending on your application.

      Reply
  2. Valeriy

    Thanks for interesting information. I wondering to know, how is it possible to control within the Matlab webcam image acquisition parameters like Exposition time, Image Brightness, Contrast etc.? Webcam driver gives such possibility, but only from his own software. Thanks for any ideas.
    Valeriy

    Reply
    1. Makarand Tapaswi Post author

      Hi
      You might want to try the following:
      vid = videoinput(‘winvideo’); % load the object
      src = getselectedsource(vid); % get properties of video object
      get(src) % gets the list of properties, also inspect(src)
      propinfo(src, ‘Brightness’) % returns the range of acceptable values
      set(src, ‘Exposure’, 1); % set specific properties

      Hopefully the get(src) will return options for your camera that you need. It is a little tricky and camera / manufacturer specific.

      Reply
    1. Makarand Tapaswi Post author

      Hi, could you provide some more details? What is the error message, etc.? This should be doable.

      Reply
    1. Makarand Tapaswi Post author

      Hi, You can save the video by using the VideoWriter class. I didn’t understand the second part of your question, what text do you want to obtain?

      Reply
  3. jerin

    sir we are getting the video input in the matlab window,now we have to save that video file.we are capturing only the lip movements.that lip movements having some informations.we have to give that information as text.how can we give sir? pls help sir pls…..

    Reply
      1. George

        Sir
        how do u plot a bounding box around the output of the background subtraction process?????

      2. Makarand Tapaswi Post author

        I am assuming you have patches of pixels where you know the background has not changed and others where it has changed. A simple way would be to find the min and max x and y coordinates of the pixels and then obtain a box from min to max.

      3. Makarand Tapaswi Post author

        Your original question is too vague. What do you want to do exactly? Do you have the patches that I assumed? What are you using for background subtraction?
        The code for what I proposed (if it applies) would be somthing on the lines of
        y1 = min(patches(:, 1)); y2 = max(patches(:, 1));
        x1 = min(patches(:, 2)); x2 = max(patches(:, 2));
        where patches is a vector of pixel locations values which are in your foreground.
        I guess you can then draw a box (see rectangle funtion).

      4. George

        im sorry but i didnt understand what u meant by patches sir.
        here is my code
        clc;
        close all;
        clear all;

        source = videoinput(‘winvideo’);
        set(source, ‘ReturnedColorSpace’, ‘RGB’);
        set(source, ‘FramesPerTrigger’, 1);
        set(source, ‘TriggerRepeat’, 50);
        triggerconfig(source, ‘manual’);
        start(source);
        thresh = 30/255;
        pause(2);
        trigger(source);

        bg = getdata(source,1,’double’);
        bg_bw= rgb2gray(bg);
        bgfilt=medfilt2(bg_bw);
        fr_size = size(bg);
        width = fr_size(2);
        height = fr_size(1);
        f = zeros(height, width);

        for i=1:100

        trigger(source);

        fr=getdata(source,1,’double’);
        fr1=fr(:,:,:,1);
        fr_bw1=rgb2gray(fr1);

        bg_fr_diff = abs((double(bgfilt)) – (double(fr_bw1)));

        for j=1:width
        for k=1:height
        if (bg_fr_diff(k,j) > thresh)
        f(k,j) =255;
        else
        f(k,j) = 0;
        end

        end
        end

        f1=medfilt2(f);

        bgfilt=fr_bw1;

        figure(1),subplot(2,1,1),imshow(fr1)
        subplot(2,1,2),imshow(uint8(f1))

      5. Makarand Tapaswi Post author

        Hi George
        I think I understand what you want now. At the bottom of your code, near the figure plotting, you can add this snippet of code.
        [yy, xx] = find(f1);
        rect_y = min(yy); rect_x = min(xx);
        rect_h = max(yy) – min(yy);
        rect_w = max(xx) – min(xx);
        rectangle(‘Position’, [rect_x, rect_y, rect_w, rect_h], ‘EdgeColor’, ‘r’, ‘LineWidth’, 2);

        The find with give you all locations of pixels which have changed. The rect_[x,y,w,h] are the bounding box coordinates which are then fed to Matlab’s rectangle function. Note that I do not do any error checking right now, so in case there is no movement, the width/height can be 0 and matlab will complain. But I guess you can fix that using a try-catch.

      6. George

        thank you very much sir.. u were crystal clear. i jst have another doubt.
        for the error checking part, when there is nothing moving is there any function that keeps the matlab running normally and not do anything??

      7. Makarand Tapaswi Post author

        Look up the try-catch exception handling in Matlab documentation. Basically the idea is everything you think is dangerous and might give errors can go in “try” and if there is an error, the code jumps to “catch” section. So in your case this might work.
        try
        rectangle(‘Position’, …);
        catch
        fprintf(‘Failed, but nothing to do, just move on.\n’);
        end

      8. Sanjeev

        thanks for that sir..
        but my bounding box is jst 1 for all moving object and not individual boxes..

        and i used the regionprops command 2 find centroid of the object. i get a min of 3 after removing the excess. how do i get only the important 1???

  4. jerin

    sir i’m doing my project in voice to indian sign language translation, i want to convert speech commands into sign symbols. could you specify the codings sir please….

    Reply
    1. Makarand Tapaswi Post author

      Hi Jerin, firstly this is a rather large project, something that would take time to think about and implement and certainly something that one does not just have the code for lying in his backyard! I am not sure what level of complexity you want to achieve, but especially if you are a student I would say you should first start off by recognizing simple and few (5 – 10?) speech commands.

      So, one possible method that comes to my mind is
      1. Speech to text
      a. Check out full-fledged ASR systems such as the HMM toolkit – HTK (might get very complicated very fast) OR
      b. Use some features (MFCC) and build some templates / models for your “keywords” (the ones you have signs for) by recording a few friends saying those words. At test time, you can align the speech to the templates using DTW (Dynamic Time Warping) and then see which word it fits best. This is rather crude, but was one of the early successful ways of doing speech recognition for single words.

      2. Display of symbols
      Again for a simple start, store the list of symbols as images. Display the image that corresponds to the recognized keyword and be done. It is not very fluid, and jumps from keyword to keyword, but should do for a nice student project.

      Making it complex would be to really do the full-fledged ASR and animate a hand to generate the symbols based on some rules? I’m assuming there are rules in sign language.

      Reply
  5. Roflo

    Hello

    On the image acquisition toolbox, when i click the start preview and then start the acquisition, the preview become nothing.

    The preview screen turned to green color with some distortion a the top.

    My camera works fine with its software

    Thank you

    Reply
    1. Makarand Tapaswi Post author

      You should try searching for whether your camera is supported by Matlab. Is it a rare model?

      Reply
  6. sachin

    while setting up webcam for matlab when i type the videoinput command i get an error.
    obj = videoinput(‘winvideo’, 1, ‘YUY_1024x768)
    |
    Error: The input character is not valid in MATLAB statements or expressions.

    Sir,please help me.

    Reply
    1. Makarand Tapaswi Post author

      There seem to be many tiny errors. (i) a missing end quote after YUV, (ii) are you sure it’s YUY? I think YU*V* is more correct. (iii) Don’t copy the quotation marks from websites, the matlab apostrophe is different from web font ‘ which is usually slanted.

      Reply
      1. Makarand Tapaswi Post author

        That might not be entirely true. Yes, YUV sounds more natural but apparently in the ‘linuxvideo’ device, I see some ‘YUYV_640x480′ and other resolutions. imaqtool is interesting to check.

  7. Malik

    Makarand Tapaswi Actually I am doing motion detection but when I connect my webcam to PC and type videoinput command it gives me error
    >>vid = videoinput(‘winvideo’, 2, ‘RGB24_320x240′);
    ??? vid = videoinput(‘winvideo’, 2, ‘RGB24_320x240);
    |
    Error: The input character is not valid in MATLAB statements or expressions.

    Please tell me whats the error as I checked by typing your suggestion YU*Y* but same error and also by typing YUV but nothing happened.please help me

    Reply
    1. Makarand Tapaswi Post author

      This might be silly of me, but can you try rewriting the command? Copy-pasting from a blog messes up the single-quote which Matlab requires. It’s a bit more slanted.

      Reply
      1. Malik

        Makarand Tapaswi Please tell how to write code for background subtraction algorithm in Matlab with a webcam interfaced?

    2. Sanjeev T George

      i think this is enuf for bg subtraction. this code comes up after setting up the webcam
      bg = getdata(source,1,’double’);
      bg_bw= rgb2gray(bg);
      fr=getdata(source,1,’double’);
      fr1=fr(:,:,:,1);
      fr_bw1=rgb2gray(fr1);

      f1= 255 * ((bg_fr_diff > thresh));

      bgfilt=fr_bw1

      Reply
      1. aseer670

        Sanjeev T George Can we do thresholding(If pixel value>threshold covert it to 255 otherwise 0) of an image after binarization?and can we find gray value of each pixel of an image after this thresholding?Please clear it to me.

      2. Sanjeev T George

        Hi asser670,
        Yes you can do thresholding by giving an IF clause and specifying the right conditions. the problem being the code is executed a little more slowly and affects performance. I actually did specify the If clause and thats when when sum1 told me to replace that particular line of code with “f1= 255 * ((bg_fr_diff > thresh));” which basically performs the same function with no challenge to the performance.
        And what do you mean finding the gray value of each pixel? Do you want to display those pixel values?

      3. aseer670

        Sanjeev T George when all the values in the matrix of image will be 1 or 0 after binarization then how we will do its thresholding,means how can we set a threshold value of it because there will be only two values 1 or 0 then what will be the threshold value which we can set and convert it to 255 or 0?And Grey scale value is a format means grey value of each pixel…

      4. STG

        For my use i set the threshold value to 20. For most cases its set between 20 and 30. There are papers online that set a value for threshold depending upon the image. that is, its set extrinsically
        I still did not understand what.is ur requirement. you could display the gray values on a graph sheet using suitable coding.,

      5. aseer670

        Dear I understand what is thresholding but i am not getting the point that after binarization(when each value will be 0 or 1)how can we set a threshold value of 20 or 30 to do its thresholding?

      6. STG

        sorry abt that….. It is 1 and 0. only thing that 255 represents the gray scale value of 1.. try replacing the 255 with 1 and see the diff.

    1. Makarand Tapaswi Post author

      Hi Rohit, I’m not sure what you mean by compare, but in case you mean difference, then just store the images in two variables and then they are just matrices of integer type. You may want to convert to double and then just subtract. Visually you can do a subplot(211); imshow(im1); subplot(212); imshow(im2); to look at the 2 images saved in variables im1 / im2.

      Reply
      1. rohitravishankar

        oh!! so the on the basis of that difference, can in tell how how similar the two images are?

      2. Makarand Tapaswi Post author

        Well that depends on what you mean by image similarity — there are 2 massive fields of work on that image processing and computer vision. As a start you could just subtract at pixel level and then put a threshold on the absolute value. Say if the pixel difference is less than 10, (or 20, heavily depends on application) you call the images similar, or not.

  8. girish

    Can i get code for background substraction using web cam real time . nd application of using this. If possible to track any individual in a crowd.can we get the movement in colour instead of gray

    Reply

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s