Re: [Paparazzi-devel] What's the state of ardrone2

-Christophe

On Mon, Dec 15, 2014 at 2:59 AM, Ori Pessach <address@hidden> wrote:

Not if you want the result to be a normal MPEG-4 video bitstream, you can't... :)

I was able to encode a larger frame size, but I suspect that the codec itself (the dll64P file that runs on the DSP) has some limitations. Still, I was able to encode an 848x480 video at 10fps, and it seemed to work fine. It actually worked perfectly as long as I saved the video to the drone's flash filesystem (using a filesink.) Streaming it over UDP has a lot of issues, but since saving the video locally works perfectly I think that points to the UDP element in gstreamer as the problem. Either that, or the client side is broken, but I used both VLC and gstreamer as clients, and both behave in the same way, so I don't think this is a client issue.

I'm still planning to get the H.264 encoder that's already on the drone to work with gstreamer, if that's possible. I think I understand enough of gst-dsp to at least give it a try.

I also want to get the mp4mux working, so I can record MP4 files during flight, and a working RTSP server would be a nice thing to have, for my own twisted purposes.

To make recording MP4 files more convenient than it currently is, it would be nice to have a module that can autodetect and mount a USB drive when one is inserted. It would also be necessary to have the recording module stop cleanly (so it can close the MP4 file and finish writing all the necessary boxes that make it playable.) The signal to do that would have to be tied to a flight event - probably something in the flight plan that happens after the drone lands. This is basically how Parrot handled recording - except that their firmware also looks for unclosed files and tidies them up, which I was actually really impressed to see. If any Parrot engineers are reading this - nice work! You saved a video from one of my more dramatic crashes, and saved me the work of trying to salvage the video myself.

Ori

On Sun, Dec 14, 2014 at 3:22 PM, Christophe De Wagter <address@hidden> wrote:
We were wondering why this dspmp4enc could not handle larger frames. We were thinking of a memory issue. Can't you send 4x 320 frames to represent 1 640 frame but allow smaller buffers?

On Dec 14, 2014 7:40 PM, "Ori Pessach" <address@hidden> wrote:
Thanks! This looks useful as a starting point for a more full featured implementation, but as it stands now it falls short of what I'm looking for.

First, it only streams low resolution MPEG4 video over UDP, whereas I'm interested in storing 720p H.264 video on a USB stick while simultaneously streaming a lower quality stream to a laptop. Even the lower quality stream shouldn't be 320x240. That's kinda bad.

There are many missing piece which I'm currently working on figuring out: the ISO-14496-12 multiplexer appears to be missing from the version of gstreamer that comes with ardrone2_vision, and there's no working H.264 encoder. The former is relatively easy to fix, while the latter seems to be a bit trickier.

I'm basically trying to get to feature parity with the Parrot firmware, at least when it comes to video support. Once I'm there, I can start looking at doing some fancier stuff...

Ori

On Sun, Dec 14, 2014 at 2:59 AM, Christophe De Wagter <address@hidden> wrote:
What you describe is not in the repo at the moment but has been done before: a simple module.xml with the following start and stop functions:

void gst_video_start(void)
{
// Start GST plugiun
// Normal video 320
system("/opt/arm/gst/bin/gst-launch v4l2src device=/dev/video1 ! videorate ! 'video/x-raw-yuv,framerate=15/1' ! videoscale ! video/x-raw-yuv, width=320, height=240 ! dspmp4venc ! rtpmp4vpay config-interval=2 ! udpsink host=192.168.1.255 port=5000 &");
}

void gst_video_stop(void)
{
// Stop GST-Plugin
system("kill -9 `pidof gst-launch-0.10` &");
}

and raw:

<raw>

upload_extra:
@echo "====================================VISION START/INSTALL======================================"
ardron2.py startvision

</raw>

we have

-Christophe

On Sun, Dec 7, 2014 at 4:53 PM, Ori Pessach <address@hidden> wrote:
Also, I couldn't figure out from the documentation how to load any modules other than the ardrone2_vision mjpeg module... It sounds like there's a way to include the ardrone2_gstreamer module in a paparazzi airframe file so that it sets everything up and starts a gstreamer pipeline when you press upload, but I couldn't figure out how to do it.

Any tips?

On Sun, Dec 7, 2014 at 7:54 AM, Christophe De Wagter <address@hidden> wrote:
ardrone2_vision contains mainly computer vision code and is work in progress.

ardrone2_gstreamer is quite ready and has what you ask for: there are a few examples of gstreamer chains and the upload button has all the needed code to upload images to the dsp and start it.

-Christophe

On Sat, Dec 6, 2014 at 10:58 PM, Ori Pessach <address@hidden> wrote:
Hi,

I'm trying to get the vision framework to work, but something is amiss. I can get the mjpeg streaming module to work (by including it in the airframe file) but the image quality is quite poor, and there's a lot of latency when viewed with VLC. I'm guessing it's streamed over a TCP connection, which can do that.

What I would like to do is try to run the H.264 encoder on the DSP using gstreamer, and save the video to a USB thumb drive on the vehicle, at the very least. This should give the drone similar functionality to what you would get with the Parrot firmware.

As a first step, I'm trying to just run a pipeline to encode MPEG4 on the drone using gst-launch, (as described here: http://lists.gnu.org/archive/html/paparazzi-devel/2013-11/msg00184.html)

This kinda works, after futzing with environment variables on the command line and stopping paparazzi (since with the MJPEG module, it grabs /dev/video1 and we can't have that now, can we?)

So, seeing that I can get gst-launch to work on the drone, I should be able to write a paparazzi module using gstreamer to run the same pipeline and use it to replace the mjpeg module. Right?

What I'm wondering about, though, is why the H.264 DSP based encoder doesn't get installed along with the MPEG-4 encoder. Is this intentional? Is it an oversight? Can I just copy it over, launch it in the same way that ardrone2.py launces the MPEG-4 encoder and expect it to work? Am I missing something?

I would love it if someone who's familiar with the code and how it's supposed to work would comment on that.

Thanks,

Ori Pessach

_______________________________________________
Paparazzi-devel mailing list
address@hidden
https://lists.nongnu.org/mailman/listinfo/paparazzi-devel

_______________________________________________
Paparazzi-devel mailing list
address@hidden
https://lists.nongnu.org/mailman/listinfo/paparazzi-devel

_______________________________________________
Paparazzi-devel mailing list
address@hidden
https://lists.nongnu.org/mailman/listinfo/paparazzi-devel

_______________________________________________
Paparazzi-devel mailing list
address@hidden
https://lists.nongnu.org/mailman/listinfo/paparazzi-devel

_______________________________________________
Paparazzi-devel mailing list
address@hidden
https://lists.nongnu.org/mailman/listinfo/paparazzi-devel

_______________________________________________
Paparazzi-devel mailing list
address@hidden
https://lists.nongnu.org/mailman/listinfo/paparazzi-devel

_______________________________________________
Paparazzi-devel mailing list
address@hidden
https://lists.nongnu.org/mailman/listinfo/paparazzi-devel

From:	Christophe De Wagter
Subject:	Re: [Paparazzi-devel] What's the state of ardrone2_vision?
Date:	Mon, 15 Dec 2014 09:15:37 +0100

Re: [Paparazzi-devel] What's the state of ardrone2_vision?