本帖最后由 飞凌-marketing 于 2022-1-27 11:00 编辑
) Q- B/ R" K) v
3 x2 W# k @; v; p6 H1 ~& B3 B' i5 V
0 C0 b( V5 \$ h4 W作者|donatello1996 来源 | 电子发烧友 题图|飞凌嵌入式 iMX8MPlus 核心板: https://www.forlinx.com/product/136.html
# |( V( P) I* p
0 R N3 _# _6 ~. k. I- S8 H本文采用的硬件板卡为飞凌嵌入式OKMX8MP-C开发板,系统版本Linux5.4.70+Qt5.15.0,主要介绍基于HTTP网页服务器和UDP上位机的MJPG码流传输。 MJPG格式作为一种持续传输的视频码流,在远程监控领域中应用较广,而实现这种远程监控的第三方应用最常见的有两种:浏览器HTTP网页、UDP上位机。 5 e! ^+ J* [$ k

7 L2 Y& w* _% l2 i, V两者各有优势,对比鲜明,其中: 这两种应用各有优缺点,对于嵌入式开发者来说,两者都必须掌握。 ! x- O! {) C! ?
一、HTTP网页服务器
K, X& x3 S5 n) r/ U; @9 I K; J先说下HTTP网页服务器获取MJPG码流的代码,首先是OKMX8MP-C在开发板端建立TCP服务器: - int TCP_Server_Found(socklen_t* socket_found , char* ip , int port)3 K9 Q6 H/ G7 r" _: H9 x2 H1 P
- {! @* q* O5 T0 D
- struct sockaddr_in servaddr;8 B1 P% _" g [5 ?" N! X* y
- socklen_t addrsize = sizeof(struct sockaddr);' k- a9 g. g# g, B1 `8 g, u
- bzero(&servaddr , sizeof(servaddr));* ?6 t) @& t% e2 ^( `
- servaddr.sin_family = AF_INET;
+ r5 s" k3 U+ @6 J& i - servaddr.sin_addr.s_addr = inet_addr(ip);6 g+ Y5 d: V" N, F0 z( I7 v' W
- servaddr.sin_port = htons(port);
. R: r2 A$ K( o; h& B - int ret;2 ^# [4 D! x* T) \+ z
- IF( (*socket_found = socket(AF_INET , SOCK_STREAM , 0)) == -1). g5 i$ y |1 G3 H
- {
0 }4 r9 g' Q p8 Z- Z2 K - printf("Create socket error: %s (errno :%d)\n",strerror(errno),errno);
9 ]0 W/ v3 I- N5 H' Y - return -1;
' Q0 C$ Y, h7 s; l; n - }
! Z4 ~* G9 B8 |1 `0 E3 {, ~ - int on = 1;' M& y' J) d9 T6 D4 Z
- if(setsockopt(*socket_found , SOL_SOCKET, SO_REUSEADDR, &on, sizeof(on)) < 0)
" t" {3 B- u! D+ ` - {
) d$ H: _2 R' E( d+ R$ o% k - printf("setsockopt error\n");
0 ^/ e) P: C5 T4 U# W& ~ - }
9 F' z& c' W- L - ret = bind(*socket_found , (struct sockaddr *)&servaddr , addrsize);
( w9 a& q- V- u - if(ret == -1)* I* Z+ ^2 U3 x9 Q; R+ K
- {
/ O! A+ k5 g& }; U - printf("Tcp bind faiLED!\n");1 `5 T. [- Q, N7 e e2 V; ]
- return -1;
n, K, Y) y- E* k3 _! ] - }
/ r: Z& w- M5 D- `0 n0 M7 l8 g - if(listen(*socket_found , 5) == -1)1 O" ~, y+ _ J1 j9 K5 Y) _
- {& {* I8 l4 z6 y) v! @
- printf("Listen failed!\n");$ G! }+ ?* p- U- k; y
- return -1;
6 _; j" W) z0 ^' r5 z( v - }! o+ L8 {: L6 e
- return 0;" b4 ]. J8 l, V- ]6 H) P
- }
复制代码其中setsockopt()函数是可选的,一般只用于规避socket()函数的建立错误。 建立了TCP服务器后,返回的socklen_t型实参在后面的HTTP网页服务器中需要用到。 HTTP网页服务器所属的TCP操作是需要另起轮询线程来让客户端进行accept()握手操作的,accept()之前的listen()倒是只需要执行一次即可,accept()握手操作和recv()接收操作需要创建一个死循环线程: - pthread_create(&tid_tcp_web_recv , NULL , Thread_TCP_Web_Recv , NULL);
" _2 S q8 N9 x7 T - void * Thread_TCP_Web_Recv(void *arg)
/ c! r. H3 v! e5 z4 t t1 f- } - {
/ A" b+ Y# v4 y3 ` - 。。。% [) L5 z [6 t5 @2 y7 H& ?) d
- while(1)$ P& z' B8 j. I& h; s
- {
( u7 Z+ i2 b1 m0 S2 w/ x' Z - fd_socket_conn = accept(socket_web_server , (struct sockaddr *)&sockaddr_in_conn , &addrsize);* ~& ~- `, j ?+ r# R
- printf("fd_socket_conn = accept()\n");( p+ {" n+ N$ k( Z3 k- ^, O5 u
- 。。。
7 \3 G7 C( Z" O! D5 S- g# _$ Y1 } - recv(fd_socket_conn , recvbuf , 1000 , 0);+ |4 x* F* ]3 K2 K
- }
/ f$ B) `, V' E- u& t2 D" i! Q - 。。。' b6 k4 q6 F* n$ v
- }
复制代码MJPG帧可以使用Grab操作获取,获取到的MJPG帧需要在TCP线程中读,在Grab操作线程中写,这种被多个线程访问的资源需要加锁防止读写冲突,即资源被Grab操作写入时,需要上锁,不允许其它线程访问,操作完成时需要解锁,允许其它线程访问: - pthread_mutex_lock(&pmt);; z8 X9 n4 j& Q7 V
- pic_tmpbuffer = pic.tmpbuffer;
! H- Z/ J* Y# h, t) J - pic.tmpbytesused = buff.bytesused;
8 W N% s' z* d' t - pic_tmpbytesused = pic.tmpbytesused;$ N8 }# c- r7 M G, a4 r; ?
- pthread_cond_broadcast(&pct);
- F& i& X% T9 I i% k; ? - pthread_mutex_unlock(&pmt);
复制代码线程互斥锁使用之前需要初始化: - pthread_mutex_t pmt; U5 D8 g& n [4 M( v' ~, R/ z
- pthread_cond_t pct;
4 m, v/ g+ r, D: l: x/ ` - int main(int argc, char* argv[])( `. `7 D8 O; H& f. M1 \
- {! a1 m2 `7 H8 y9 `
- ...
% |$ z7 K$ q9 t: H9 @8 U3 `/ B, g. O - TCP_Server_Found(&socket_web_server , (char*)argv[2] , PORT_TCP);
/ o) s- ^2 u. o- n/ V3 _7 d - pthread_mutex_init(&pmt , NULL);, ?) P, o5 O8 g- U' I
- pthread_create(&tid_tcp_web_recv , NULL , Thread_TCP_Web_Recv , NULL);
8 ?6 n/ c d7 l - pthread_create(&tid_tcp_web_send , NULL , Thread_TCP_Web_Send , NULL);
5 L; u" y0 O* w9 f6 r( E/ L- t - ...
4 n; p S, b/ X+ q6 K% ^ - while(1)6 y$ t9 X& Y& ^# m; [& N
- {. d. u$ W9 T% R8 c
- V4l2_Grab_Mjpeg(false , MJPEG_FILE_NAME);
. `; E, F0 J o7 \$ ^ {, @1 c - ...
$ s7 c( X5 L5 C u - }- v ~& y! L q/ J" `! d7 H# A
- ...5 r( i5 n3 X4 f T2 G/ H/ b
- }
复制代码然后是发送的细节,发送图片文件之前,需要先发送HTTP标准头,这个相当于给发送图片或者其它类型的流数据铺路: - <p style="box-sizing: border-box; border: 0px; vertical-align: baseline; line-height: 26px;"><span style="text-indent: 32px;">
$ U6 O k9 J( d) Z& @ - </span></p><p style="box-sizing: border-box; border: 0px; vertical-align: baseline; line-height: 26px;"></p>
复制代码- #define STD_HEADER "Connection: close\r\n" \8 Z" r/ i" g) V+ L+ T( v, H
- "Server: MJPG-Streamer/0.2\r\n" \
2 J2 J" v/ J$ ] - "Cache-Control: no-store, no-cache, must-revalidate, pre-check=0, post-check=0, max-age=0\r\n" \9 S/ o. R8 ^& Z/ z8 }
- "Pragma: no-cache\r\n" \/ @8 _$ l: R" S/ ]% e
- "Expires: Mon, 3 Jan 2000 12:34:56 GMT\r\n"" a6 W: S$ H& C* T: g, h
- #define BOUNDARY "boundarydonotcross"
* Z) P# P% |; C9 I4 { - printf("preparing header\n");
- q4 v% z( N9 S; u1 H0 j6 e - sprintf(buffer, "HTTP/1.0 200 OK\r\n" \
6 Z( o. d6 R. P8 p4 S* c m! { - "Access-Control-Allow-Origin: *\r\n" \
4 Z# ~: _! N, n# s; U - STD_HEADER \1 s) \: |# o" D: E7 G z. A7 V/ b
- "Content-Type: multipart/x-mixed-replace;boundary=" BOUNDARY "\r\n" \
4 y! h/ Y h: n3 g+ d( W - "\r\n" \9 L9 e. E$ ?& d% n' R e
- "--" BOUNDARY "\r\n");4 H# @: U' a2 C% h4 z
- if(write(fd, buffer, strlen(buffer)) < 0)) l" o7 D" m) Q# ~4 X R
- {* _* |& k& {4 [2 W
- free(frame);8 z) I; J/ t) M2 W, f
- return;
7 |+ B9 @& @7 s - }
复制代码发送完HTTP标准头之后,就需要发送内容头(Content-Type),这处的Content-Type为image/jpeg,同样,HTTP标准协议里面image支持的类型远不止jpeg一种,发送完内容头之后就是正文和boundary结尾,这样帧完整的HTTP头发送到指定的TCP GET地址,就会在浏览器中显示刚刚发送的图片: - <pre class="prettyprint lang-cpp" style="box-sizing: border-box; font-family: Monaco, Menlo, Consolas, "Courier New", monospace; font-size: 16px; white-space: pre-wrap; line-height: 1.38462; color: rgb(51, 51, 51); word-break: break-all; background-color: rgb(245, 245, 245); border: 0px; border-radius: 4px; vertical-align: baseline;"> sprintf(buffer, "Content-Type: image/jpeg\r\n" \
( z$ w! A9 W! @8 m - "Content-Length: %d\r\n" \
$ Y' F9 H# D% Q v; D' N& ~8 o - "X-Timestamp: %d.%06d\r\n" \6 Q3 X3 s; A/ n5 e$ U3 V
- "\r\n", frame_size, (int)timestamp.tv_sec, (int)timestamp.tv_usec);& h! Z% n9 x. A/ I; A( c
- printf("sending intemdiate header\n");9 W& A! N# V2 A$ o
- if(write(fd, buffer, strlen(buffer)) < 0)- I0 N! S: ~8 Y }
- break;
. L+ A: G( `" V5 J9 v6 m - printf("sending frame\n");( g3 F" F$ Z0 ]$ F( E( m; D' M
- if(write(fd, frame, frame_size) < 0) L O& M6 V3 E. D1 r1 o1 z
- break;
9 C `% U1 o! w% ~ l& B% u( Z - printf("sending boundary\n");/ T; ?% U6 v$ Z- |+ T2 b' o$ c3 ?
- sprintf(buffer, "\r\n--" BOUNDARY "\r\n");
4 G$ ^; L3 V1 f! U - if(write(fd, buffer, strlen(buffer)) < 0)
; i5 x' n' q1 o' b; A3 X - break;</pre><p style="box-sizing: border-box; border: 0px; font-size: 16px; vertical-align: baseline; line-height: 26px; color: rgb(51, 51, 51); font-family: " helvetica="" neue",="" helvetica,="" tahoma,="" arial,="" "microsoft="" yahei",="" "hiragino="" sans="" gb",="" "wenquanyi="" micro="" hei",="" sans-serif;"=""></p>
复制代码另外需要说明的是,TCP服务器线程在发送MJPEG流的时候是死循环发送的,因此TCP客户端在发送完GET指令之后,就会收到TCP服务器循环发送的图像缓存,TCP客户端会陷入忙等待状态无法再对外发送任何GET或者POST指令,从客户端使用者角度来看的效果就是网页一直在等待。 
: b2 t7 G( k4 y+ n: s) n + \( V) d9 c6 d' L5 L, z
二、UDP上位机UDP发送操作,同样需要先建立UDP Socket:3 o- p4 v8 _/ w# m( P! z
- int UDP_Send_Found(socklen_t* socket_found , struct sockaddr_in *addr , char* ip , int port)
; } y D' _2 Q; l - {* l& P( a4 h# j- z1 [! C
- *socket_found = socket(AF_INET, SOCK_DGRAM, 0);
- {, w D5 Y8 B# E6 P9 q - if(*socket_found == (~0))
. s9 `( t7 E7 G! w+ H! z - {
& p7 s" ~, i* l - printf("Create udp send socket failed!\n");
- X( h, a0 K8 t9 D2 x) K/ n' q - return -1;$ J8 ~; Y1 u7 n2 ?& ^
- }
7 V- r. g4 }9 H0 [5 M - addr->sin_family = AF_INET;5 z' l- D! C. g8 J9 D
- addr->sin_addr.s_addr = inet_addr(ip);% A, l$ @* F: e- C( H* r; {) {
- addr->sin_port = htons(port);' z7 i @0 }0 |) P. r! ?
- memset(addr->sin_zero, 0, 8);
' a2 U9 z4 {: s2 c4 W% X8 { - return 0;
7 ? i* `" M7 p3 t) j* r x - }
复制代码
3 J0 I& t+ @# l4 J l- O% v/ \& D1 f* w1 c. i
而UDP文件发送则要比HTTP发送简单得多,只需要将文件切片,每一片为固定长度的UDP帧长度,逐帧发送即可:
% X4 w4 w, V$ M5 J, r; |$ R* }& o% v2 P' i
9 k9 L% v: j) W8 Z) [
- while(fend > 0)7 U* F! c% V+ N" p5 K/ R+ b
- {
) M* e8 p$ `+ N. Z% l3 \ - memset(picture.data , 0 , sizeof(picture.data));) o; _% I, k7 B0 [
- fread(picture.data , UDP_FRAME_LEN , 1, fp);; o. O; ^6 s! q
- if(fend >= UDP_FRAME_LEN)
]- A+ u) d {6 G3 b0 H - {
2 {/ L" C, r% H7 L( s- P. c - picture.length = UDP_FRAME_LEN;; A; ]& ?+ [$ g% R# X1 s$ w C
- picture.fin = 0;* D$ T4 m) R2 C! l% q
- }
" }7 b7 {) E' c2 o - else ~" D6 }* I/ M) W5 W
- {
+ ~7 V9 B4 l, {0 `2 ~% d, p - picture.length = fend;
0 G5 O6 ?' q6 \) p - picture.fin = 1;( r8 z( L9 m6 A+ h+ `) ]
- }
! w. Z A, O/ B# v9 m - //printf("sendbytes = %d \n",sendbytes);
7 { I {% M+ h# @# } M( I+ n - sendbytes = sendto(socket_send, (char *)&picture, sizeof(struct Package), 0, (struct sockaddr*)&addr,addr_len); ?0 ]0 W0 L2 p, d# h0 q+ p
- if(sendbytes == -1)
& \( _. r2 t0 y5 ?3 O+ d - {
$ E- g, O+ a, ]& j! q - printf("Send Picture Failed!d\n");: n% ]; L3 e O
- return -1;
- R4 E5 e, Y; v0 m+ V$ Z7 _! e* Y - }
% C6 E. g3 g2 J2 p% O - else
. l% R; S+ t6 w# s( m - { O4 K# x/ w- i y! |% p/ e3 i; P5 y
- fend -= UDP_FRAME_LEN;7 @! s6 H) g" `$ D
- }% \1 Q5 i9 M8 \6 U3 w2 ~/ u
- }
复制代码
4 `0 ^! y& Y d3 a: ? V
+ G% \% J+ N& E, m7 X0 Z s
$ ?, ^: R6 z( m+ V/ K" ^- q
M4 W0 ]4 W2 n' V: {: YiMX8MPlus 核心板: https://www.forlinx.com/product/136.html |