NEW ROS GoPro Hero Package

/

To date, an official ROS package for the GoPro Hero does not exist, and I plan for this package (https://github.com/Seanmatthews/gopro_hero) to remedy that. Currently only tested with the GoPro Hero 4 Black, the nodes within allow:
* Access to camera settings
* Taking and retrieving still images (including multishot)
* Continuous live stream frame retrieval

We plan to attach the GoPro Hero 4 Black camera to the sub's external housing. A WiFi dongle (separate from the sub's router), plugged into the sub's vision computer, connects to the GoPro's network. That computer accesses the GoPro's stream for logging and processing. The network configuration here seems (and is) somewhat complex, but the GoPro requires that it is the LAN provider that allows devices to connect to it for control and streaming. Briefly, we considered bridging the primary sub network to the GoPro's network to avoid an additional network and network device, but the GoPro's network is fairly unstable and we don't want the sub relying on it in any capacity.

The GoPro Hero 4's live stream comes from a UDP stream (udp://10.5.5.9:8554). Any connection must command the GoPro to restart the stream after it connects, and then send stay alive messages every 2 to 2.5 seconds. A similar effect may be achieved by sending the stream restart command at the same interval, but this method produces a video blip on each restart command. 

The package is written as a collection of C++ nodes because 1) I'm more comfortable with the language, and 2) I initially investigated using ffmpeg to grab still frame images from the GoPro's UDP stream (though that proved troublesome), with the expectation that it could be tuned for minimal latency. While I still plan to explore this avenue, it turns out that OpenCV's VideoCapture class is perfectly capable of capturing frames from the live stream with only a small amount of latency. 

During development, I drew on a number of sources that paved the way for this node. Some even implemented their own GoPro control programs and APIs. Should you opt to write your own GoPro control program, here they are:
* https://github.com/Seanmatthews/goprowifihack
* https://www.reddit.com/r/gopro/comments/2md8hm/how_to_livestream_from_a_gopro_hero4/
* https://gist.github.com/3v1n0/38bcd4f7f0cb3c279bad#file-hero4-udp-keep-alive-send-py
* http://ffmpeg.org/

Stream to PC from GoPro Hero4 Black

/

One of our recent design improvements is the addition of a GoPro Hero4 Black. Admittedly, I bought it without much research because the thing looks friggin awesome. I mean, look at that beaut...

Pros:

  • Small, fits in the palm of your hand
  • 4k @~30fps, and more fps for less resolution
  • Modes out the wazoo, including low-light
  • Waterproof to ~140ft

Cons:

  • < 1hr battery life
  • Lack of public API
  • Pricey
  • No wired streaming
  • Noticeable latency

When it comes to streaming, GoPro provides only one option-- mobile devices. They've made it very clear that the primary use case, in their eyes, remains on-device recording and playback. However, they left open a REST interface for all commands and settings, almost undoubtedly intentionally. Find all of those commands here. 

Finally, the steps I followed to stream the Hero4 to my Mac. The specific commands will vary between OSes, but the general instructions are the same.

  1. Install ffmpeg & ffplay. On OS X, that's brew install ffmpeg --with-ffplay
  2. Install GoPro app on your mobile device, use it to setup wireless, and stream video to that device. That last part is important, as I believe it initializes something important.
  3. Connect to your GoPro's wireless network.
  4. Back out of all menus on the GoPro device. I've noticed the device may not be able to stream when it's in a menu.
  5. Open your computer's firewall to port 8554 for UDP traffic.
  6. In a terminal, ffplay udp://:8554
  7. In a web browser, go to http://10.5.5.9/gp/gpControl/execute?p1=gpStream&c1=restart. You should see the response {"stream" : "0"}.

When you flip over to the ffplay window, you should see a streaming video from your GoPro with very high latency (~6s). To remedy this to some extent, stop ffplay, and repeat steps four and five, but replace step four with ffplay -fflags nobuffer -f:v mpegts -probesize 8192 udp://:8554. This should bring the latency down to less than 2s.

Note that when the Hero4 Black first came out, GoPro release it with a different firmware, whose commands were different. If the above steps don't work for you, check your firmware version against the latest.

Bonus: Underwater!

Our use case for this camera was going to travel one of two paths. Either we'd simply attach it to the outside of the sub and press record before letting it loose, or we'd also stream video back to one of the computers for real-time image processing. Since the GoPro Hero4 Black acts as a wireless access point, we thought we could potentially connect to it, through ~1ft of water, from inside the sub. Turns out it works in the bathtub (i.e. not salt water) to just under a foot!

References:
  1. https://www.reddit.com/r/gopro/comments/2md8hm/how_to_livestream_from_a_gopro_hero4/cr1b193
  2. https://www.reddit.com/r/gopro/comments/2md8hm/how_to_livestream_from_a_gopro_hero4/coh94hi