本帖最后由 飞凌-marketing 于 2022-1-27 11:00 编辑
, \' y7 ^, W# q% b$ }! r" ^8 P, N+ [1 c9 p9 ]8 ^# E, b

t- h. l- x' F$ i3 M作者|donatello1996 来源 | 电子发烧友 题图|飞凌嵌入式 iMX8MPlus 核心板: https://www.forlinx.com/product/136.html7 S: `! k V+ N j- I$ A% j
/ J8 j# S }: R, j. k8 _
本文采用的硬件板卡为飞凌嵌入式OKMX8MP-C开发板,系统版本Linux5.4.70+Qt5.15.0,主要介绍基于HTTP网页服务器和UDP上位机的MJPG码流传输。 MJPG格式作为一种持续传输的视频码流,在远程监控领域中应用较广,而实现这种远程监控的第三方应用最常见的有两种:浏览器HTTP网页、UDP上位机。 " ^$ J% I. D. f- F3 d

' v& e" W$ b, M1 O, s" J7 g* X两者各有优势,对比鲜明,其中: 这两种应用各有优缺点,对于嵌入式开发者来说,两者都必须掌握。 4 a& t% K$ T% o! |
一、HTTP网页服务器) ~2 D0 U" X, t, K
先说下HTTP网页服务器获取MJPG码流的代码,首先是OKMX8MP-C在开发板端建立TCP服务器: - int TCP_Server_Found(socklen_t* socket_found , char* ip , int port)
/ Q9 s( ^1 M0 S& @2 m' K4 C9 a6 O - {8 o/ m8 l% R5 ^ l7 D" [. P
- struct sockaddr_in servaddr;) e* I+ J( ?3 O' g, t. L9 Y$ V& O
- socklen_t addrsize = sizeof(struct sockaddr);! E: Y3 D8 X: y. g8 n, _
- bzero(&servaddr , sizeof(servaddr));# d+ G3 x/ _) h5 l
- servaddr.sin_family = AF_INET;, @& z% k: w. m- G
- servaddr.sin_addr.s_addr = inet_addr(ip);3 |) W& x9 D+ |' N: c
- servaddr.sin_port = htons(port);
( a8 I' z" u9 g8 O' D - int ret;: z% _& C! q" F) @
- IF( (*socket_found = socket(AF_INET , SOCK_STREAM , 0)) == -1)
1 M* X# H- A0 X2 T3 V; j5 ~ - {+ R. A- Z- y) |9 D. T% `. f
- printf("Create socket error: %s (errno :%d)\n",strerror(errno),errno);, n% T7 @- p6 \% ~/ o
- return -1;/ k* z2 `# X2 n" [9 ]: ^' W
- }
% a, e% B _1 Y) Y5 x+ u2 ?0 n) i - int on = 1;3 c! D! u1 ] Q" }" k' r2 [& e* p
- if(setsockopt(*socket_found , SOL_SOCKET, SO_REUSEADDR, &on, sizeof(on)) < 0)$ b. R; z+ b) D1 y
- {
* e: d0 \/ w, z2 U! h. Y4 n - printf("setsockopt error\n");
* V k. g7 e# D G. E - }. l9 [4 ?2 Y$ B X2 C& x+ d: b* j
- ret = bind(*socket_found , (struct sockaddr *)&servaddr , addrsize);- {6 ?2 _, ? o: Q, t3 S
- if(ret == -1)
I& {- z6 H5 q7 r$ D+ F& l - {$ N, K2 ~6 H" F- V0 E B+ {
- printf("Tcp bind faiLED!\n");
' {; k" t5 K7 @5 N- C - return -1;) y5 ?& d8 Y2 r ?" U1 X) p
- }
+ H5 U- r- [- I$ j7 |8 W1 q2 l - if(listen(*socket_found , 5) == -1)
, m6 S0 t l6 K7 D% @$ `+ A - {
- ~: G; ~* Q: {$ h- L - printf("Listen failed!\n");+ X4 q6 a; G. R
- return -1;
8 R+ H7 U, O4 j - }
4 I8 M) Q4 r6 b9 K0 b$ U/ @$ |5 w5 u - return 0;
5 P, U# D k5 n' @0 ^! Z - }
复制代码其中setsockopt()函数是可选的,一般只用于规避socket()函数的建立错误。 建立了TCP服务器后,返回的socklen_t型实参在后面的HTTP网页服务器中需要用到。 HTTP网页服务器所属的TCP操作是需要另起轮询线程来让客户端进行accept()握手操作的,accept()之前的listen()倒是只需要执行一次即可,accept()握手操作和recv()接收操作需要创建一个死循环线程: - pthread_create(&tid_tcp_web_recv , NULL , Thread_TCP_Web_Recv , NULL);
+ }' s5 I* a* E& V( s2 e4 b% u+ ^3 N - void * Thread_TCP_Web_Recv(void *arg)
& V5 y: M E- {3 I& q - {' E X! X: S4 j. x, d! f* @
- 。。。
1 b( e* s# z2 [* y& p - while(1)" }+ `, a/ l- y1 X) B. r' e
- {
$ y0 a8 ^9 l3 A6 r/ y: I0 x - fd_socket_conn = accept(socket_web_server , (struct sockaddr *)&sockaddr_in_conn , &addrsize);7 [* y9 H5 G4 s' t( x
- printf("fd_socket_conn = accept()\n");) N3 M7 h) X; k/ p
- 。。。
- Y% m& C; w% s6 z6 e* X/ H - recv(fd_socket_conn , recvbuf , 1000 , 0);# h* w3 |! c ]" D% L
- }' T( o, y3 L% x" [* a6 E
- 。。。0 o* U: H$ g6 ?5 K' x1 j
- }
复制代码MJPG帧可以使用Grab操作获取,获取到的MJPG帧需要在TCP线程中读,在Grab操作线程中写,这种被多个线程访问的资源需要加锁防止读写冲突,即资源被Grab操作写入时,需要上锁,不允许其它线程访问,操作完成时需要解锁,允许其它线程访问: - pthread_mutex_lock(&pmt);& d2 L% i/ C' B2 N/ n) V
- pic_tmpbuffer = pic.tmpbuffer;
# h5 ~2 k* k6 e, b5 U - pic.tmpbytesused = buff.bytesused;
, P5 d8 ~% L z9 G& z* t$ k1 f( _ - pic_tmpbytesused = pic.tmpbytesused;
% j. y9 d* M/ F8 l! N+ d - pthread_cond_broadcast(&pct);
4 N% D9 D4 z S: M+ C T% u - pthread_mutex_unlock(&pmt);
复制代码线程互斥锁使用之前需要初始化: - pthread_mutex_t pmt;) p4 w$ [4 j! s6 w
- pthread_cond_t pct;. C6 L6 v+ g& A4 G* c; c* }
- int main(int argc, char* argv[])' a% A( s" c- M" \- Z
- {
$ w5 ]3 F3 n3 b - ...& C$ W; I) `/ D' n4 S3 Z% {7 M
- TCP_Server_Found(&socket_web_server , (char*)argv[2] , PORT_TCP);
; ^7 _* B( X2 H8 q) r) b - pthread_mutex_init(&pmt , NULL);
" ^) `" z+ M1 x1 ]6 r* q - pthread_create(&tid_tcp_web_recv , NULL , Thread_TCP_Web_Recv , NULL);
+ H8 S2 H& E8 m0 z - pthread_create(&tid_tcp_web_send , NULL , Thread_TCP_Web_Send , NULL);
; ~( t$ d7 Z; s. T+ {6 I - ...
9 ^3 B+ Q% X: K h2 A) x) d: E - while(1)
- J3 b4 a) |. D- N5 q - {7 X+ }" ~: d; b; v8 V
- V4l2_Grab_Mjpeg(false , MJPEG_FILE_NAME);
Y3 H/ | |( ]8 C. r( J - ...$ |: W# m3 R1 ^: N
- }4 M) a/ U7 `; j
- ...5 x G) h9 ?4 {" s5 P) Q& E: L, T9 u
- }
复制代码然后是发送的细节,发送图片文件之前,需要先发送HTTP标准头,这个相当于给发送图片或者其它类型的流数据铺路: - <p style="box-sizing: border-box; border: 0px; vertical-align: baseline; line-height: 26px;"><span style="text-indent: 32px;">
. G* F& [8 d1 [, k8 k% R* ^ - </span></p><p style="box-sizing: border-box; border: 0px; vertical-align: baseline; line-height: 26px;"></p>
复制代码- #define STD_HEADER "Connection: close\r\n" \* w# { A/ W& o' H6 v$ @# y% R
- "Server: MJPG-Streamer/0.2\r\n" \
% I, ~$ a# q/ ^! K# U }& {/ \0 _ - "Cache-Control: no-store, no-cache, must-revalidate, pre-check=0, post-check=0, max-age=0\r\n" \6 A) B2 Y" c# _; B- i: l
- "Pragma: no-cache\r\n" \% r& `! h; Z1 v$ ]; A0 ]: C4 [
- "Expires: Mon, 3 Jan 2000 12:34:56 GMT\r\n"1 e7 L" J3 M0 ]* M; f: R" Q
- #define BOUNDARY "boundarydonotcross"
. z, D3 z7 b6 u1 I @, |: X( Q, i' C3 a- Z - printf("preparing header\n");
) x! Q3 v2 {- i# A! ^) F - sprintf(buffer, "HTTP/1.0 200 OK\r\n" \
& ~" I4 T( z3 F2 \% ^ - "Access-Control-Allow-Origin: *\r\n" \
3 n5 D# }* j2 g, I; B, n K+ | - STD_HEADER \
* I2 C1 ~' @, s! f8 ~4 k+ S8 Z7 D - "Content-Type: multipart/x-mixed-replace;boundary=" BOUNDARY "\r\n" \
8 y6 ?* a# [/ _$ b- e3 X - "\r\n" \
( Y- w! ~7 ?5 [' M8 `* Z# l( s - "--" BOUNDARY "\r\n");) }5 |4 _4 U" C" M8 f
- if(write(fd, buffer, strlen(buffer)) < 0)
6 w/ {/ \4 D9 e6 [$ } D - {. X9 }% y' S [* P
- free(frame);
8 R; n, n/ p" `5 I" |2 I1 q - return;
( O1 o, P) U, K6 K" i ~& U - }
复制代码发送完HTTP标准头之后,就需要发送内容头(Content-Type),这处的Content-Type为image/jpeg,同样,HTTP标准协议里面image支持的类型远不止jpeg一种,发送完内容头之后就是正文和boundary结尾,这样帧完整的HTTP头发送到指定的TCP GET地址,就会在浏览器中显示刚刚发送的图片: - <pre class="prettyprint lang-cpp" style="box-sizing: border-box; font-family: Monaco, Menlo, Consolas, "Courier New", monospace; font-size: 16px; white-space: pre-wrap; line-height: 1.38462; color: rgb(51, 51, 51); word-break: break-all; background-color: rgb(245, 245, 245); border: 0px; border-radius: 4px; vertical-align: baseline;"> sprintf(buffer, "Content-Type: image/jpeg\r\n" \4 }8 B) c4 ]2 v$ P$ X$ D7 ]2 C6 z5 ~
- "Content-Length: %d\r\n" \+ S m' I4 `; G: x0 T/ B
- "X-Timestamp: %d.%06d\r\n" \0 W5 r# k4 x+ B" }; m1 [
- "\r\n", frame_size, (int)timestamp.tv_sec, (int)timestamp.tv_usec);0 _) P; ]; {# `/ w7 e: M8 L2 @
- printf("sending intemdiate header\n");! v2 I. k2 d( M# [' G( R: Q
- if(write(fd, buffer, strlen(buffer)) < 0)2 o5 K% b- O$ t& u" C
- break;4 \2 q% H7 U0 d7 _# `0 D9 V5 R! k
- printf("sending frame\n");0 {8 ^- X/ N7 i( Z2 Y
- if(write(fd, frame, frame_size) < 0)
7 ~( I% k) Y2 w4 o/ f# f - break;
1 I8 m5 i# k! u) M- ]. r - printf("sending boundary\n");
4 N( Z, D: h$ A {! W - sprintf(buffer, "\r\n--" BOUNDARY "\r\n");- v; l1 A, v/ G# w/ }2 R# r
- if(write(fd, buffer, strlen(buffer)) < 0)
Q* y2 `* T# B) z6 U) F G - break;</pre><p style="box-sizing: border-box; border: 0px; font-size: 16px; vertical-align: baseline; line-height: 26px; color: rgb(51, 51, 51); font-family: " helvetica="" neue",="" helvetica,="" tahoma,="" arial,="" "microsoft="" yahei",="" "hiragino="" sans="" gb",="" "wenquanyi="" micro="" hei",="" sans-serif;"=""></p>
复制代码另外需要说明的是,TCP服务器线程在发送MJPEG流的时候是死循环发送的,因此TCP客户端在发送完GET指令之后,就会收到TCP服务器循环发送的图像缓存,TCP客户端会陷入忙等待状态无法再对外发送任何GET或者POST指令,从客户端使用者角度来看的效果就是网页一直在等待。  5 O, y. z p: z E$ T v$ Y
 8 h" Y( V \$ Q3 i" {
二、UDP上位机UDP发送操作,同样需要先建立UDP Socket:
" ^) N x2 t! f2 P- int UDP_Send_Found(socklen_t* socket_found , struct sockaddr_in *addr , char* ip , int port)) y: e0 [' E& r* T
- {" F, _3 K* |2 w& u2 u* d+ d* q
- *socket_found = socket(AF_INET, SOCK_DGRAM, 0);, h2 j- s) ]+ r( l/ I! _4 z
- if(*socket_found == (~0))( W1 }3 |5 P; q# S% P2 T% U
- {
% x9 Y; l! h) [( A - printf("Create udp send socket failed!\n");# q1 a r& O, b! g9 b
- return -1;& O* c1 f4 N) c# U& J
- }3 c8 S9 i6 V9 g+ q" _; e: q
- addr->sin_family = AF_INET;
4 S9 G0 R" H2 w1 r- f - addr->sin_addr.s_addr = inet_addr(ip);
* h6 f8 w" f( T - addr->sin_port = htons(port);
9 s3 C' e; {# i. ^1 R - memset(addr->sin_zero, 0, 8);
* J* j5 `! q9 O) j - return 0;
I5 x: g7 O# |% T7 a3 s, B - }
复制代码
! z! q; K$ N( k) Q0 K% g5 y; H$ B" e$ E4 A& X
而UDP文件发送则要比HTTP发送简单得多,只需要将文件切片,每一片为固定长度的UDP帧长度,逐帧发送即可:
[% L& v( i, ^: Y. f9 r* N, A) R; y K V( P
7 S$ Y: @7 a5 q2 d, k6 a$ w- while(fend > 0)) w: A7 ?8 I' L; g1 N" _9 r
- {0 o" @' p2 G5 P. O7 \; j1 D9 x
- memset(picture.data , 0 , sizeof(picture.data));
g9 E) G& m: i7 R3 @6 ?, L4 t; k) s - fread(picture.data , UDP_FRAME_LEN , 1, fp);% H) p0 r) X$ O
- if(fend >= UDP_FRAME_LEN)
3 a q1 M2 ?; z5 Z8 l$ S0 q5 F$ A - {
9 [, N: {+ o2 {2 X - picture.length = UDP_FRAME_LEN;
8 m9 ], C% X! Y8 P0 w. F) y: B - picture.fin = 0;
7 t7 S3 T6 B" h0 L) P0 g+ }: W - }; l3 b7 j6 N( k5 d% h
- else
/ }# z$ K1 T4 O, _# Y# d - {, {5 M3 Y% U% W
- picture.length = fend;( Z" X: X/ X2 ^ t
- picture.fin = 1;
( W0 p1 d' y% r3 D( H5 R0 j - }
( W+ t( K, l5 E+ V+ B% k - //printf("sendbytes = %d \n",sendbytes);
& k, x: O% ^6 j s. Z4 {5 x# A. q0 ] - sendbytes = sendto(socket_send, (char *)&picture, sizeof(struct Package), 0, (struct sockaddr*)&addr,addr_len);3 w) j' z& N* `- C/ A3 V
- if(sendbytes == -1)
5 E ]# b- K+ z! W& }5 Z - {
; b4 E* q! t+ A& B6 e" c8 z - printf("Send Picture Failed!d\n");
8 `% T& N- Y# ~ - return -1;4 n5 j. y* P2 R% D
- }/ @' b) p0 {' y1 o4 P$ }0 e
- else2 F) g. l& V* ]
- {
/ v5 S: W1 d! I - fend -= UDP_FRAME_LEN;
3 s! ^% W; h. a( ?; J: w' R" A - }( N8 S7 o" a2 E* O. a
- }
复制代码
$ g0 Y: A: e) |) f+ l0 ]( D) ]0 q) O& D' _; J; O3 U! D

C; c" d' F: X' S0 ]9 `, ~) S+ Y" {: A+ s" r
iMX8MPlus 核心板: https://www.forlinx.com/product/136.html |