IP cameras and vb6 1

[tedsmith]

Yes I'm still alive, my 78th birthday is next week!
Has anybody and suggestions as to how to capture frames from an IP camera connected to the computer?
I would want these to be in a format that can be sent say one per second on a local network to other workstations for display on standard image boxes like my previous applications using USB web cameras that I have listed on this forum. They could be bitmaps of jpgs.

There are a number of progs about that use special ocx's to show IP video in a special video box but I was hoping to use ordinary vb6 controls and some APIs if this is possible.

It appears that most IP cameras send a mp4 stream.

 

[dilettante]

IP Cameras use a number of different protocols, some built on top of TCP and some on top of UDP. Almost all of the ones I saw browsing the Web can use HTTP, many use one of a couple of transport protocols designed just for IP cameras.

On top of that you have other variations. Often they support still-image retrieval, live streams, and recorded streams.

Others have PTZ (pan, tilt, zoom) hardware and either use a special IP camera transport supporting those control operations as well as still and video retrieval or else they may use a separate control protocol using a second TCP or UDP port. They may also offer remote control for UV filter enable/disable, IR lamp control, device reset, and maybe a lot more.

And some also capture live audio while others allow you to send audio the other way (to a speaker at the camera).

Still images seem to always be JPEG, but video and audio formats can vary quite a bit. MP4 seems to be a low-quality oddball format a few use. It looks more common in the market right now to use H.264 for reduced bandwidth and M-JPEG for higher quality and more "admissable" video.

Quite a few IP cameras appear to support multiple streams, usually two but sometimes three. As far as I can tell the different streams can be for things like a high quality stream, a low quality stream, and a "playback recorded video" stream at the same time. Or one stream can be set for one video format and another stream for another format. Lots of them seems to be able to record to on-board flash memory, while others are meant to have one channel of video used by a multi-camera DVR device for central recording.

 

[dilettante]

I didn't see any IP camera that will just stream video frames over raw TCP. You need something else in order to pass information about the frame's format or at least each frame's size in bytes.

So the odds of finding one that you can just use a naked Winsock control with are pretty slim. However I'm no expert, I've only read through the spec sheets of a handful of IP cameras in the low-end to mid-range categories and a few articles published by the security camera "CCTV" industry.

I had mentioned HTTP because it is one of the most common protocols IP cameras appear to support for retrieving both still frames and video.

Your best bet for still image retrieval in VB6 seems to be to use the built in AsyncRead method which can directly retrieve and return a StdPicture object to you. You might also use the WinHttpRequest object but then you will need to convert the bytes received into a StdPicture using WIA 2.0, some OLE API calls, or some GDI calls...

... or the crude aproach: write the bytes to disk, then re-read them back using LoadPicture().


The crudest possible way might be to use a 3rd party program or a control designed to retrieve and display IP camera video and then attempt to "scrape" the visible window to grab frames from it. You could do it... but WOW what an awkward approach! Probably one that eats the most CPU possible and the one most likely to flake out and lock up or even crash.

 

[strongm]

I'm happy to stick with my "frankly not from VB6, and in general not without wrappers " ...

 

[dilettante]

Well if one insists on MP4 streams (which seem to be obsolete for IP cameras) or H.264 streams (which pretty much displaced the use of MP4 as far as I can tell) I think you're correct that you're stuck with DirectShow. I'm not sure the necessary codec filters are readily available either. The other alternative might be a vendor SDK providing a library, but the two I downloaded only offer iOS and Android code and expect you to handle this for Windows, etc. yourself from their docs.


From my reading, it looks like the older M-JPEG approach is still very viable despite its higher bandwidth requirements. Mainly for legal reasons (MPEG-4 and H.264 seem to be considered useless as forensic evidence due to serious compression artifacts).

I have a cobbled-together VB6 program that uses a Winsock control and a few OLE calls to retrieve M-JPEG streams and JPEG stills, and it seems to work fine. All it really needs to have done to it is the addition of HTTP Digest authentication, more thorough testing, and some profiling to find spots worth optimizing. In other words the dull and time consuming stuff.

I don't have any local IP cameras but it seems to work fine pulling a 640x480 M-JPEG stream from a camera in Sweden at 7 to 15 frames/sec (limited by Internet congestion end-to-end as far as I can tell).

 

[strongm]

640 X 480? Ted stated he wants to go with ip cameras because the resolution of webcams is too low. And since current low end USB web cams will happily do 720p I can only assume he wants something closer to 1920 * 1080. Be interesting to see what sort of M-JPEG frame rate you might get with that.

 

[dilettante]

While I agree higher resolution and even wide-aspect IP cams are available, even just above the very low end, I'm not sure that's what he really needs. If so, I would think you reach diminishing returns fast beyond 720p though.

So far I haven't found any public cameras online doing M-JPEG at more than 640x480. Maybe I need to do some more searches.

I wouldn't be surprised if his problem with USB webcams isn't more a question of optics than resolution. The majority are fixed-focus at 2 feet for video chat and a lot use a single plastic lens. I'd expect more fixed-focus IP cameras to be set at about 30 feet, with optical glass lenses or even compound lenses being more common. And many come with adjustable focus.

M-JPEG at 720p 4:3 aspect (960x720) would be pushing it. Even that could require a ton of bandwidth, so for most purposes 1080p M-JPEG would probably be out of the question. Most of them appear to allow the JPEG compression level to be changed though and that could make a difference - but then you trade off quality again.

Of course even considering MPEG-4 or H.264 means you don't care about image quality, and if there is a lot of motion in the field of view you lose the comperssion advantage but retain the high costs. Those are not "cheap" streams to encode or to decode so both ends of the process suffer (camera electronics overheat and then fail early).

Dropping the frame rate to 1 or 2 fps makes almost all of the disadvantages of M-JPEG disappear, yet MPEG-4 and H.264 image quality gets even worse and the bandwidth goes up again unless the field of view is completely static.

I guess until I can put an IP camera on my LAN I won't really know what is feasible. I assume he wants to be able to have multiple live feeds on-screen at once.

 

[tedsmith]

Sorry, when I mentioned why use an IP camera, rather than the resolution, I wanted an improvement in "quality".
Because they are mostly designed for indoor views of faces in a bedroom (web cams). While most new webcams give high definition anyway, they perform poorly when used in high contrast situations, Ceiling lights bloom badly when the exposure is adjusted so you can see any detail in the shadows. Headlights of vehicles bloom really badly.

Comparing my HD Sony handycam with the Microsoft Lifecam on 1900 x 1024, they both give the same sharpness of picture but the Sony gives a vastly better picture in every other respect. (It should as it cost over $1000 compared with $85 for the webcam). The transducers and lenses are different for a start.

I will try the same method I used in receiving a JPG from a remote image box using GDI that I used in my previous project but as you say this will depend on the camera (that will be unfortunately be chosen by others).

My client will be lending me a IP camera in a few months time and I'll let you know if it works.

 

[dilettante]

I got curious and ran some tests.

My program was eating 7% to 10% of the CPU fetching and displaying that 640x480 M-JPEG stream from Sweden at 13 fps. This is a 7 year old Intel Q6600 machine, so call it 40% of one 2.4GHz core.

Then I ran some tests with the VB Watch profiler. Almost all (seriously, nearly 95%) of the CPU time used is being spent on a single statement: a call to OleLoadPicture() to convert a Byte array of JPEG frame data to a StdPicture for display! I had not anticipated that.

This tells me three things:
[ul]
[li]There isn't much left worth optimizing here except possibly to look for a faster JPEG decoder DLL to use.[/li]
[li]Something like 1920x1080 may be impossible except at low frame rates.[/li]
[li]Attempting to use a more "advanced compression" stream encoding format is going to
burn CPU like crazy, since they are well known to be costly to decode and render.[/li]
[/ul]

However I'm not a multimedia guy and I have to admit I don't know what to expect of good hardware accelerated codecs.

 

[tedsmith]

Can you tell me the address of the Swedish camera so I can test too? I understand their girls are at least as good looking as Aussie girls.

 

[tedsmith]

Just had another idea.
Maybe there is existing software that could run independently of my app and put the pics from the IP camera on the clipboard.
My existing app interrogates the clipboard every second and looks for pics from the old USB camera using AviCap.Dll
If there was such a program, changes to my app would be minimal.
I'll do a bit of searching.

 

[dilettante]

The view shown through that camera isn't especially sensational (though perhaps part of that is the season?). It's a shot between two flagpoles with waving flags down some sort of paved commons area between two streets, looking toward a government building in the near distance. You can see some traffic signals change, walking people, moving cars and their head and taillights. Cold skies and bare trees.

It is possible some prebuilt software might copy frames one by one to the clipboard. I haven't seen that in the usage instructions for the stock application shipped with the two vendors' products I've browsed through.

A rate of one frame per second shouldn't make performance issues critical for you, so one less concern there.

 

[Error7]

Ted, there's a few public IP cameras here:

http://www.webcamxp.com/publicipcams.aspx

[gray]Experience is something you don't get until just after you need it.[/gray]

 

[dilettante]

That was useful. Found one of the links took me to a page that I was able to extract the direct M-JPEG feed URL from.

I had no trouble streaming 1280x720 at a solid 6.7 fps. The interesting thing is that the program doesn't use any more CPU to run than for the Swedish 640x480 stream (however the frame rate is only half that of the camera in Sweden). I'll have to compare a few frames from different sources. I'm sure they have different JPEG quality/compression levels set.

 

[Error7]

Here's a few more to play with (can't guarantee that they all still work)
142.163.209.60
142.240.200.10:1205
193.253.98.130:2222
195.54.250.138:1080
205.174.168.132
207.81.24.114:5555
209.169.157.162
209.169.162.28
209.226.48.74:81
212.219.113.227
217.155.151.110
217.41.42.231
66.102.69.149:12121
80.177.205.41:8080
81.149.71.48
84.96.70.172
194.240.224.195
213.216.201.204
87.139.11.4
212.204.48.2
141.28.126.76
212.66.14.135
83.218.185.13:81
217.86.174.228:8081
81.201.199.106
62.225.134.156
88.116.117.219
81.92.14.130
130.83.13.228
213.16.181.191
213.5.235.220:85
89.148.93.13
85.119.9.21:2004
193.4.101.156
157.157.216.141
194.144.85.204
86.43.102.46:8080
82.90.14.219
82.88.111.195:1080
88.41.58.162
82.188.195.100
89.97.241.232
213.178.199.158:8888
81.73.229.182
62.225.134.156
88.53.197.250
88.32.225.4
85.20.134.146:88
220.56.208.4
200.67.209.172:60000
201.120.30.228

[gray]Experience is something you don't get until just after you need it.[/gray]

 

[dilettante]

Ripping out the OleLoadPicture() code and substituting WIA 2.0 I got a 37% improvement in CPU use and what subjectively appears to be much better JPEG decoding (images look sharper).

WIA 2.0 uses GDI+ 1.1, at least on Vista and later. I have no idea how that works if you deploy the WinXP/redist version of WIA 2.0, but XP is dead now anyway.

I'll have to watch a bit more. Supposedly GDI+ 1.1 has a problem with monochrome JPEG decoding, creating dithered results. Common IP cameras supposedly go "night vision" at a certain light level, eliminating their IR filters and switching to monochrome.

 

[dilettante]

While all of this was interesting to play with, in the end you might want to look at SDKs for the IP camera application industry.

Here is one that offers a nice ActiveX control that should easily be usable in VB6 programs:

AXIS Media Control

Note that for MPEG-4 or H.264 support you may have to jump through some additional licensing hoops because of patents on those technologies. Probably one more point in favor of M-JPEG.

 

[tedsmith]

Thanks for the info.
I will pursue this when I get a my hands on a camera in a few weeks time.

 

[strongm]

So, more or less back to what Error7 suggested 11 Mar 14 15:17

 

[dilettante]

Pretty much. Though handling M-JPEG in pure VB is perfectly feasible. Add some API calls and you can even record it to AVI files.

Funny part is one of my clients read this thread and is paying me to finish the thing I was working on for their use because... they rejected the Axis control. It works fine as far as I can tell but they insist on having source after being seriously burned when a 3rd party .Net library disappeared on them a few years back.

Hard to blame them, look at what has happened to VB6 3rd party libraries over the years.

 

[dilettante]

Hmm, while I'm at it I should probably mention another possibility that might be harder to "stumble over" depending upon your Google-Fu.

Instead of something like that Axis SDK you might look at a free M-JPEG-over-HTTP DirectShow source filter Roman Ryltsov wrote and made available. This link should take you to all of his related blog posts, which contain download links. Both 32-bit and 64-bit filters are available as well as HTTPS support. In the future he may get around to adding H.264-over-HTTP as well but it looks like he has moved on to other projects.

Here is the link: Fooling Around tag M-JPEG

The downside is that he hasn't built in workarounds for cameras that fail to properly implement this. I've found about 3 such oddities myself and all of them come down to errors in implementing the Content-Type used (oops, I wasn't supposed to mention that). ;-)

Roman's fallback is to ignore the camera's M-JPEG stream URL and just pull individual JPEGs frame by frame at low rates from its still image URL instead. Perfectly feasible for some applications.

Aside from that quirk it should be a lighter weight alternative to most vendor SDKs. Sadly the source is not available, but that won't matter for a lot of people anyway.