本帖最后由 飞凌-marketing 于 2022-1-27 11:00 编辑
! F' ^5 N/ {2 K) ~
8 s* ^6 d* v8 G" I* _- a % O8 f; U7 [& M+ w; D& w/ a
作者|donatello1996 来源 | 电子发烧友 题图|飞凌嵌入式 iMX8MPlus 核心板: https://www.forlinx.com/product/136.html
" T& k2 A$ |+ b1 T' ^" n4 _- f6 X- U0 v7 p7 Z2 g
本文采用的硬件板卡为飞凌嵌入式OKMX8MP-C开发板,系统版本Linux5.4.70+Qt5.15.0,主要介绍基于HTTP网页服务器和UDP上位机的MJPG码流传输。 MJPG格式作为一种持续传输的视频码流,在远程监控领域中应用较广,而实现这种远程监控的第三方应用最常见的有两种:浏览器HTTP网页、UDP上位机。 ' F7 y! f, \7 k6 U
 # ^5 b4 W0 M9 M1 C( d0 `4 t" t
两者各有优势,对比鲜明,其中: 这两种应用各有优缺点,对于嵌入式开发者来说,两者都必须掌握。 * B# }# v9 W& p& w k
一、HTTP网页服务器+ y% h2 u6 i2 n# e
先说下HTTP网页服务器获取MJPG码流的代码,首先是OKMX8MP-C在开发板端建立TCP服务器: - int TCP_Server_Found(socklen_t* socket_found , char* ip , int port)
1 s* L8 A2 i/ G, m1 f e# X( O - {
7 U1 C: k9 J H ~8 J/ o, L! V( ` - struct sockaddr_in servaddr;! I' h" O& x; F' ^
- socklen_t addrsize = sizeof(struct sockaddr);
4 g( u9 |' L( c" u+ A - bzero(&servaddr , sizeof(servaddr));
% p9 l0 U; f" |# [, N4 H - servaddr.sin_family = AF_INET;* s0 k' x8 W: t. C" y( N
- servaddr.sin_addr.s_addr = inet_addr(ip);/ n5 S0 r. _/ k
- servaddr.sin_port = htons(port);
; C1 y. t' d/ X9 ]0 U7 l6 d - int ret;5 {# C* W: g/ W8 U1 i6 c# D# W
- IF( (*socket_found = socket(AF_INET , SOCK_STREAM , 0)) == -1)
$ @' ~( S* q( t - {
; h% H! }6 M9 }+ ~# t6 F+ [ - printf("Create socket error: %s (errno :%d)\n",strerror(errno),errno);
* T: y- Q+ i4 d+ g - return -1;
+ M9 }5 a1 V3 W* \7 H6 Y - }% r7 Z/ }9 y# x- L) Z
- int on = 1;
; n- A$ M( X+ I% h# _# U) B - if(setsockopt(*socket_found , SOL_SOCKET, SO_REUSEADDR, &on, sizeof(on)) < 0)
# V; X% z% w% F: e$ {, A - {$ @( X! J" L6 q/ J6 p* I7 L
- printf("setsockopt error\n");
8 a% N" x: j( j0 r - }
% |! R/ R' I: k/ Q% \" V& l - ret = bind(*socket_found , (struct sockaddr *)&servaddr , addrsize);
" r; w! E; L4 n! {: t5 i3 I - if(ret == -1)& p/ ^* V: B9 h2 F# h4 V
- {" U$ N) x5 S0 Z% C6 O
- printf("Tcp bind faiLED!\n");
; `- w" C+ |8 n) ~ - return -1;
* ~9 z& j" V0 U* C: B9 T9 X - }) M' v x. I* u+ X
- if(listen(*socket_found , 5) == -1)( v+ M3 N* Y1 a: @) P# o C- L
- {) ]- p8 s8 \# H( q8 ^
- printf("Listen failed!\n");
; Y! Y4 y+ V3 e- |2 a* k! j0 R - return -1;
5 q# g6 J9 E, e2 B- @1 S - }( ` Z) D- ^0 w6 n* v/ n
- return 0;
c% q. a; e! @+ D - }
复制代码其中setsockopt()函数是可选的,一般只用于规避socket()函数的建立错误。 建立了TCP服务器后,返回的socklen_t型实参在后面的HTTP网页服务器中需要用到。 HTTP网页服务器所属的TCP操作是需要另起轮询线程来让客户端进行accept()握手操作的,accept()之前的listen()倒是只需要执行一次即可,accept()握手操作和recv()接收操作需要创建一个死循环线程: - pthread_create(&tid_tcp_web_recv , NULL , Thread_TCP_Web_Recv , NULL);
. r3 `$ b ?/ i- K - void * Thread_TCP_Web_Recv(void *arg)
. D0 [. y* P9 c8 O: A6 D1 ] - {1 t% T, Z2 G# {3 H6 a; w
- 。。。7 ]6 `; k6 h( x6 q
- while(1)
# ^; s: }0 n' } - {
" U: \* Y+ v) ?+ F! U& `, [& o( i - fd_socket_conn = accept(socket_web_server , (struct sockaddr *)&sockaddr_in_conn , &addrsize);/ l) [# J2 e6 u: c G& B5 K
- printf("fd_socket_conn = accept()\n");
$ c% g/ W8 O7 t2 O4 ~, R( G2 } - 。。。 D* D. a d4 t8 Y" V. p
- recv(fd_socket_conn , recvbuf , 1000 , 0);
5 Y- b3 p' y u' ~& g - } \2 ~( k: H& E4 E" J
- 。。。
1 K8 E9 `6 o2 ?8 T+ v - }
复制代码MJPG帧可以使用Grab操作获取,获取到的MJPG帧需要在TCP线程中读,在Grab操作线程中写,这种被多个线程访问的资源需要加锁防止读写冲突,即资源被Grab操作写入时,需要上锁,不允许其它线程访问,操作完成时需要解锁,允许其它线程访问: - pthread_mutex_lock(&pmt);
* J; i6 H. N. ?8 d3 }6 i - pic_tmpbuffer = pic.tmpbuffer;5 o2 d( `/ R8 z* y1 S$ q w
- pic.tmpbytesused = buff.bytesused;& g' e3 |! j7 p0 R6 @
- pic_tmpbytesused = pic.tmpbytesused;
2 ~, K0 _# j- O1 h" I - pthread_cond_broadcast(&pct);) ]+ e& y1 U' Q b( l$ L4 e
- pthread_mutex_unlock(&pmt);
复制代码线程互斥锁使用之前需要初始化: - pthread_mutex_t pmt;
% _ I5 F/ B) ^* d8 r: M% y3 U - pthread_cond_t pct;
6 R$ f/ u- U0 f; V - int main(int argc, char* argv[])
2 s S2 q7 z8 o. m( @, p- x, M - {* [0 M% d! n# K* G/ }! a
- ...2 v' [& y# o# |. F/ I$ o3 Z, ?$ s
- TCP_Server_Found(&socket_web_server , (char*)argv[2] , PORT_TCP);
6 \( N) b4 C6 y# E# k - pthread_mutex_init(&pmt , NULL);
9 y: `+ J3 E Y6 j; ? - pthread_create(&tid_tcp_web_recv , NULL , Thread_TCP_Web_Recv , NULL);
* F) i* p$ ]5 M" [& M - pthread_create(&tid_tcp_web_send , NULL , Thread_TCP_Web_Send , NULL);; E* W1 v% x* f/ S# M. w* m
- ...
; f4 Y3 C5 t, I/ {$ i - while(1)
7 c. L% `* X) k+ w$ n( [+ n - {
4 |. v4 U: _ T- L9 |6 {5 @ - V4l2_Grab_Mjpeg(false , MJPEG_FILE_NAME);
; k5 g8 [3 @1 N9 E, S& l" @3 w - ...
4 E* W2 d; P3 Z+ m0 n& E) d& } - }
: q8 B2 M( M9 ?5 g1 X' D9 t - ...
. a1 h/ S0 Q3 l - }
复制代码然后是发送的细节,发送图片文件之前,需要先发送HTTP标准头,这个相当于给发送图片或者其它类型的流数据铺路: - <p style="box-sizing: border-box; border: 0px; vertical-align: baseline; line-height: 26px;"><span style="text-indent: 32px;">+ H% B% j( A3 D0 J
- </span></p><p style="box-sizing: border-box; border: 0px; vertical-align: baseline; line-height: 26px;"></p>
复制代码- #define STD_HEADER "Connection: close\r\n" \
% ^! t8 e# ~8 w5 \3 }4 ~ - "Server: MJPG-Streamer/0.2\r\n" \. c8 g0 y3 D) u3 ]" P9 n
- "Cache-Control: no-store, no-cache, must-revalidate, pre-check=0, post-check=0, max-age=0\r\n" \! a$ s' w( h* e$ @0 |
- "Pragma: no-cache\r\n" \
# j; o% e( y* m1 [# L% c$ P - "Expires: Mon, 3 Jan 2000 12:34:56 GMT\r\n"
' y. Q- S& ~1 s! X6 n0 j - #define BOUNDARY "boundarydonotcross"9 u+ s: X2 J( i1 _9 I, e7 s
- printf("preparing header\n");& E+ q' ~' o6 A) {! u4 k
- sprintf(buffer, "HTTP/1.0 200 OK\r\n" \
# `# |8 F$ I1 d - "Access-Control-Allow-Origin: *\r\n" \9 l5 Y: L( P g/ j O
- STD_HEADER \, c' n" h' _4 D) B( U Y
- "Content-Type: multipart/x-mixed-replace;boundary=" BOUNDARY "\r\n" \
( T5 }" @& p6 u( ~0 w7 M+ N - "\r\n" \
# g1 X7 F1 _4 k; ~ - "--" BOUNDARY "\r\n");
4 J; J- I: J" m - if(write(fd, buffer, strlen(buffer)) < 0)
9 h. W& f" t5 b. L# m( W - {
4 V0 N& A* o$ R% t1 _% z+ ?0 k - free(frame);
) F8 G& E8 ~5 d* p L - return;. T4 ^- H5 t0 d% g# K0 t/ I
- }
复制代码发送完HTTP标准头之后,就需要发送内容头(Content-Type),这处的Content-Type为image/jpeg,同样,HTTP标准协议里面image支持的类型远不止jpeg一种,发送完内容头之后就是正文和boundary结尾,这样帧完整的HTTP头发送到指定的TCP GET地址,就会在浏览器中显示刚刚发送的图片: - <pre class="prettyprint lang-cpp" style="box-sizing: border-box; font-family: Monaco, Menlo, Consolas, "Courier New", monospace; font-size: 16px; white-space: pre-wrap; line-height: 1.38462; color: rgb(51, 51, 51); word-break: break-all; background-color: rgb(245, 245, 245); border: 0px; border-radius: 4px; vertical-align: baseline;"> sprintf(buffer, "Content-Type: image/jpeg\r\n" \
% s( o4 ]8 D! a) s7 C% c+ |- [' B - "Content-Length: %d\r\n" \
( k2 M9 J7 _0 y; Q) s1 r1 M - "X-Timestamp: %d.%06d\r\n" \' z" S# u, x5 T5 u8 |
- "\r\n", frame_size, (int)timestamp.tv_sec, (int)timestamp.tv_usec);
4 u# W U' J2 |" ` - printf("sending intemdiate header\n");. [- ?0 `: W- C3 M
- if(write(fd, buffer, strlen(buffer)) < 0): B: N. o0 P V
- break;
$ z8 v% G% `$ f0 h; \ - printf("sending frame\n");7 |; {% [* H: p
- if(write(fd, frame, frame_size) < 0)+ M9 W. I7 a8 k5 K6 B: i/ p, q; y
- break;- L, D7 a* k# Y( U, D! G% ]! d& a
- printf("sending boundary\n");; Q. R3 R- N( ~' V* ?6 m
- sprintf(buffer, "\r\n--" BOUNDARY "\r\n");
/ ]+ M4 X5 P. w W( E6 T* p - if(write(fd, buffer, strlen(buffer)) < 0)& x4 G8 _4 T: J0 i
- break;</pre><p style="box-sizing: border-box; border: 0px; font-size: 16px; vertical-align: baseline; line-height: 26px; color: rgb(51, 51, 51); font-family: " helvetica="" neue",="" helvetica,="" tahoma,="" arial,="" "microsoft="" yahei",="" "hiragino="" sans="" gb",="" "wenquanyi="" micro="" hei",="" sans-serif;"=""></p>
复制代码另外需要说明的是,TCP服务器线程在发送MJPEG流的时候是死循环发送的,因此TCP客户端在发送完GET指令之后,就会收到TCP服务器循环发送的图像缓存,TCP客户端会陷入忙等待状态无法再对外发送任何GET或者POST指令,从客户端使用者角度来看的效果就是网页一直在等待。 
1 V6 ` n4 Z1 R6 q5 [6 }% s3 U 1 I5 n2 R& z5 l+ P# u6 C" C
二、UDP上位机UDP发送操作,同样需要先建立UDP Socket:
1 m, W! r6 Q( e8 d' ]) q- int UDP_Send_Found(socklen_t* socket_found , struct sockaddr_in *addr , char* ip , int port)
& |7 Q, z6 T6 Y7 I/ i9 @! S4 ], m - {" n8 N' A8 ^) \
- *socket_found = socket(AF_INET, SOCK_DGRAM, 0);
" ~+ i8 e t" V - if(*socket_found == (~0))8 J h, u7 H, U
- {# z6 @0 }# j u7 m
- printf("Create udp send socket failed!\n"); `9 S. z. k. z1 `) X- @
- return -1;2 U+ n H/ g* p( m! w
- }4 O/ ~) V ]. \$ @- O; R/ L2 X
- addr->sin_family = AF_INET;
- C6 z& j" _; f0 A. I; `. t: l - addr->sin_addr.s_addr = inet_addr(ip);- {) B) x, i2 i7 z- u/ D P
- addr->sin_port = htons(port);- B' n" b/ E+ D, Y1 W, K
- memset(addr->sin_zero, 0, 8);" X0 ^3 w1 v; |+ u5 `4 |* Q0 A2 Z
- return 0;" d2 a7 k+ B1 o! i, `. C
- }
复制代码 - i4 \5 n+ v( g0 ], U3 o' F# C
( P* N( h( R. ?# F$ _
而UDP文件发送则要比HTTP发送简单得多,只需要将文件切片,每一片为固定长度的UDP帧长度,逐帧发送即可:
# y1 Q: b, y' L3 ~4 X" p* Y, V! D& `1 x- f5 ` Q
} l. |! W" B) k- while(fend > 0)1 b8 j# J/ i5 Q
- {' l4 `) h( f: G
- memset(picture.data , 0 , sizeof(picture.data));
0 o* g, H; d6 p6 r* f: W% ? - fread(picture.data , UDP_FRAME_LEN , 1, fp);) ]! _! L8 s& Q
- if(fend >= UDP_FRAME_LEN)( N4 g2 [9 q. ?0 f4 S$ e
- {
) O% e" v' w1 A# I - picture.length = UDP_FRAME_LEN;
, D( j# N+ S* x. C* [ - picture.fin = 0;
: }% v5 i% f1 s7 z# O' ~* D3 i - }+ c. C$ G8 A! X: h0 \0 @
- else
; _; [( I! E* V! }! R - {! C6 S$ x" s% v6 T9 z) M( Y4 C
- picture.length = fend;
6 q2 h5 R' e, F) n7 o9 A* V - picture.fin = 1;
; W- \/ e5 p0 n/ A - }% a0 A$ c* a2 M3 o. [ ~- T
- //printf("sendbytes = %d \n",sendbytes);
2 H% j3 t: b% R m' o- h - sendbytes = sendto(socket_send, (char *)&picture, sizeof(struct Package), 0, (struct sockaddr*)&addr,addr_len);
$ Y9 `- T. l5 B5 M" M - if(sendbytes == -1)- g3 w, s7 v. ]4 s
- {
+ k7 H. W; w6 M6 ~& K - printf("Send Picture Failed!d\n"); F, T; l% g ?# H) B( ]
- return -1;
" y$ c! X: |" n9 }1 b; p' f - }% y8 U, t- A2 B0 \8 y; k
- else
' X% B) v& [4 `8 s7 v2 F/ Y - {
* Y0 e6 p7 G9 j4 i - fend -= UDP_FRAME_LEN;# b; h8 ~" n0 S! k$ @
- }, E- e0 j! M4 v2 \9 J' F
- }
复制代码
. b( h: }8 X2 _* ^4 s; m" S: w+ e% M& w$ r/ @

; |6 ]/ N1 H4 k, D& \% w/ ]: c& r
iMX8MPlus 核心板: https://www.forlinx.com/product/136.html |