本帖最后由 飞凌-marketing 于 2022-1-27 11:00 编辑
/ a% M4 n) t& g! \
! {# U% k& p1 Z
0 D' y- g; n$ W3 h) H作者|donatello1996 来源 | 电子发烧友 题图|飞凌嵌入式 iMX8MPlus 核心板: https://www.forlinx.com/product/136.html4 g4 }+ K9 s/ a9 e' J
, C+ H6 N) \& |- v
本文采用的硬件板卡为飞凌嵌入式OKMX8MP-C开发板,系统版本Linux5.4.70+Qt5.15.0,主要介绍基于HTTP网页服务器和UDP上位机的MJPG码流传输。 MJPG格式作为一种持续传输的视频码流,在远程监控领域中应用较广,而实现这种远程监控的第三方应用最常见的有两种:浏览器HTTP网页、UDP上位机。 1 R% X. H1 b" Y3 n8 A4 K3 m7 X
 3 J7 |( R" j. _/ c
两者各有优势,对比鲜明,其中: 这两种应用各有优缺点,对于嵌入式开发者来说,两者都必须掌握。 ) l3 D& G7 E+ L! K
一、HTTP网页服务器. m+ y9 A6 v, x+ I
先说下HTTP网页服务器获取MJPG码流的代码,首先是OKMX8MP-C在开发板端建立TCP服务器: - int TCP_Server_Found(socklen_t* socket_found , char* ip , int port)/ q, c6 i3 K- c% ~& e8 c
- {
1 D. u1 w8 c" i4 u5 R+ j2 D" q& p - struct sockaddr_in servaddr;: {, z0 d# V6 f& ~2 a
- socklen_t addrsize = sizeof(struct sockaddr);9 n2 X+ B4 Y+ e$ i
- bzero(&servaddr , sizeof(servaddr));
1 S7 {: W1 f& E1 u - servaddr.sin_family = AF_INET;
8 i' N. i/ g: d b0 G B - servaddr.sin_addr.s_addr = inet_addr(ip);& W7 a7 j( r; y( U
- servaddr.sin_port = htons(port);" o( ], ^0 I. x. x$ W& H8 D8 I
- int ret;
( }4 \& ?3 F0 z- ^1 `/ P - IF( (*socket_found = socket(AF_INET , SOCK_STREAM , 0)) == -1), }4 Z( Z$ @: N
- {! V: x; R/ i, _3 Q7 F
- printf("Create socket error: %s (errno :%d)\n",strerror(errno),errno);
; W+ y% G7 }3 j* R - return -1;9 d! o+ @ W, Z2 V' c; p+ t
- }7 z: Z& m7 S/ K2 s3 }
- int on = 1;
3 f) H; c, N+ e" | - if(setsockopt(*socket_found , SOL_SOCKET, SO_REUSEADDR, &on, sizeof(on)) < 0)# j# h; g4 {6 `7 n& l
- {3 g% c$ w$ K2 i Q1 A6 C2 }8 G
- printf("setsockopt error\n");; e& W- E: z3 l" Q% |/ ]' h1 {! P
- }7 n3 G. I4 N/ z1 b! G( d
- ret = bind(*socket_found , (struct sockaddr *)&servaddr , addrsize);
1 F& T, E0 h3 D% Z - if(ret == -1)
r% v; C, H+ s3 U - {# A& w2 P4 i1 o
- printf("Tcp bind faiLED!\n");
9 H T( W8 N" {. [( `4 j: z - return -1;! l; I/ ~1 v, x" G
- }
7 y4 Y3 H$ B3 [5 s0 u3 x - if(listen(*socket_found , 5) == -1). O! D9 z6 ?& M: f. U E6 Q2 N
- {5 {. n# }6 D) n3 W' T. B/ o
- printf("Listen failed!\n");! a/ }- G' \0 ~
- return -1;
4 m# \$ `7 R6 |% i' A) F - }
; o$ Z% ]% m5 y |! K - return 0;2 |+ }5 ~) O1 V. _0 r8 x9 d
- }
复制代码其中setsockopt()函数是可选的,一般只用于规避socket()函数的建立错误。 建立了TCP服务器后,返回的socklen_t型实参在后面的HTTP网页服务器中需要用到。 HTTP网页服务器所属的TCP操作是需要另起轮询线程来让客户端进行accept()握手操作的,accept()之前的listen()倒是只需要执行一次即可,accept()握手操作和recv()接收操作需要创建一个死循环线程: - pthread_create(&tid_tcp_web_recv , NULL , Thread_TCP_Web_Recv , NULL);
8 _9 X% ~1 L; k# ?- z' D, Y - void * Thread_TCP_Web_Recv(void *arg)
9 ?* a" u5 ]% P - {9 N7 y3 e2 R2 a+ Y" {& R y; D1 o
- 。。。
* L$ x: d# G# z0 r1 T - while(1)
& ?% v3 N# Z- _+ a$ x - {5 ~, J3 N* W g7 l( T" F9 d1 }
- fd_socket_conn = accept(socket_web_server , (struct sockaddr *)&sockaddr_in_conn , &addrsize);
& i& e6 p; q% F! j - printf("fd_socket_conn = accept()\n");4 r2 x' P; A- N. P) i! N3 k4 l
- 。。。
# ^9 `) O$ g# h& }) q) W8 j - recv(fd_socket_conn , recvbuf , 1000 , 0);1 L* }1 e3 W, d
- }
$ z9 {( z$ Z! Q; U - 。。。) F4 I6 v+ q9 g* a+ J: C
- }
复制代码MJPG帧可以使用Grab操作获取,获取到的MJPG帧需要在TCP线程中读,在Grab操作线程中写,这种被多个线程访问的资源需要加锁防止读写冲突,即资源被Grab操作写入时,需要上锁,不允许其它线程访问,操作完成时需要解锁,允许其它线程访问: - pthread_mutex_lock(&pmt);
! C' {9 X5 X. E" C L/ S* Y - pic_tmpbuffer = pic.tmpbuffer;
' M# |* u! k P2 A! @8 S3 ^ - pic.tmpbytesused = buff.bytesused;5 j9 c$ P7 j- T) U1 T' M2 U& l* C
- pic_tmpbytesused = pic.tmpbytesused;4 w6 k* o+ K0 r9 L* ^1 H0 c
- pthread_cond_broadcast(&pct);: J% Z. }! n# _5 G& |4 i9 F+ J6 G. v
- pthread_mutex_unlock(&pmt);
复制代码线程互斥锁使用之前需要初始化: - pthread_mutex_t pmt;; M4 Q% j6 @: H1 q
- pthread_cond_t pct;% R& l0 v. J, P+ ]8 T- ^
- int main(int argc, char* argv[])
0 B7 X: i8 V- [; N - {
% r8 Z, A/ X* g0 a3 |0 @' @ - .../ Y0 c) t+ m# T6 A
- TCP_Server_Found(&socket_web_server , (char*)argv[2] , PORT_TCP);
# v4 ?8 ^& ]# P# }6 }) w; g - pthread_mutex_init(&pmt , NULL);4 W- t/ j. E1 l3 V6 S
- pthread_create(&tid_tcp_web_recv , NULL , Thread_TCP_Web_Recv , NULL);; k* x% g6 W: c% _% Z
- pthread_create(&tid_tcp_web_send , NULL , Thread_TCP_Web_Send , NULL);% u- x' Y7 S# N8 P- \* H1 K
- ...
7 D* N# E: t1 M1 {5 l- c# \ - while(1)
+ I/ T z# F/ c% @& |* M. M( c# N* @ - {
' d8 U9 H/ t- T7 Y& m - V4l2_Grab_Mjpeg(false , MJPEG_FILE_NAME);+ p0 Y8 H9 ?0 y1 ]0 ?5 X% v X
- ...
N/ J2 _& u# x7 r8 s - }1 x, H: l8 r# ]8 J$ }& ]
- ...
Y9 v$ G# ]! V7 Z* p$ C3 }* I; K1 c - }
复制代码然后是发送的细节,发送图片文件之前,需要先发送HTTP标准头,这个相当于给发送图片或者其它类型的流数据铺路: - <p style="box-sizing: border-box; border: 0px; vertical-align: baseline; line-height: 26px;"><span style="text-indent: 32px;">
, V8 _5 O" U9 p! |$ i, O - </span></p><p style="box-sizing: border-box; border: 0px; vertical-align: baseline; line-height: 26px;"></p>
复制代码- #define STD_HEADER "Connection: close\r\n" \* {2 B3 b. i& v
- "Server: MJPG-Streamer/0.2\r\n" \: ?( V0 F$ O4 Q, V0 f$ l: y3 i* j
- "Cache-Control: no-store, no-cache, must-revalidate, pre-check=0, post-check=0, max-age=0\r\n" \
~2 A$ {8 |1 ^ C2 E - "Pragma: no-cache\r\n" \
J# o1 _: M7 F# B - "Expires: Mon, 3 Jan 2000 12:34:56 GMT\r\n"" B6 o( I {. _/ P! R) g
- #define BOUNDARY "boundarydonotcross"
' H& V' |1 {; [ - printf("preparing header\n");4 X' c0 f$ y4 m) k% _
- sprintf(buffer, "HTTP/1.0 200 OK\r\n" \
9 w4 w/ E) o1 f - "Access-Control-Allow-Origin: *\r\n" \' E% I! i3 ]* d. I& T; q; J+ K) g1 l7 Y
- STD_HEADER \9 ^9 g2 R- b* w' A( H. ?' ~' O6 [
- "Content-Type: multipart/x-mixed-replace;boundary=" BOUNDARY "\r\n" \ h9 r+ A8 A! r
- "\r\n" \
X7 z/ `: ?: Z8 K$ ` - "--" BOUNDARY "\r\n");2 Q% H$ Q0 E: k9 {
- if(write(fd, buffer, strlen(buffer)) < 0)' r+ f7 h0 n* r% X& @( A' B
- {
' g% u, y" U. N8 z - free(frame);
4 z ?" n* I7 s- r - return;2 W: ]& m" f6 ~' K* k
- }
复制代码发送完HTTP标准头之后,就需要发送内容头(Content-Type),这处的Content-Type为image/jpeg,同样,HTTP标准协议里面image支持的类型远不止jpeg一种,发送完内容头之后就是正文和boundary结尾,这样帧完整的HTTP头发送到指定的TCP GET地址,就会在浏览器中显示刚刚发送的图片: - <pre class="prettyprint lang-cpp" style="box-sizing: border-box; font-family: Monaco, Menlo, Consolas, "Courier New", monospace; font-size: 16px; white-space: pre-wrap; line-height: 1.38462; color: rgb(51, 51, 51); word-break: break-all; background-color: rgb(245, 245, 245); border: 0px; border-radius: 4px; vertical-align: baseline;"> sprintf(buffer, "Content-Type: image/jpeg\r\n" \7 u4 e8 p% E. @" Z
- "Content-Length: %d\r\n" \* F4 A/ P" {$ B& A7 w
- "X-Timestamp: %d.%06d\r\n" \ P- p, \& v8 t
- "\r\n", frame_size, (int)timestamp.tv_sec, (int)timestamp.tv_usec);9 _( }1 e( k. S$ v6 ?1 h& s* k
- printf("sending intemdiate header\n");
2 G! G) H1 u, Y! E: t5 e, u! [ - if(write(fd, buffer, strlen(buffer)) < 0)
3 [: m* s" z8 L7 X( P! u9 Y3 y - break;! M2 |' q& Z' s9 e9 F
- printf("sending frame\n");' w+ y- ? o) i* p0 g' Y5 E, l
- if(write(fd, frame, frame_size) < 0)
- h0 W9 Z1 s; v* Q1 @ - break;1 ^# l h5 O% W# a! C& f9 r
- printf("sending boundary\n");
. p' l% {* r6 j7 `' r - sprintf(buffer, "\r\n--" BOUNDARY "\r\n");# D% H7 U% Z9 R+ z
- if(write(fd, buffer, strlen(buffer)) < 0)% V* f, |! C$ A1 I7 y N
- break;</pre><p style="box-sizing: border-box; border: 0px; font-size: 16px; vertical-align: baseline; line-height: 26px; color: rgb(51, 51, 51); font-family: " helvetica="" neue",="" helvetica,="" tahoma,="" arial,="" "microsoft="" yahei",="" "hiragino="" sans="" gb",="" "wenquanyi="" micro="" hei",="" sans-serif;"=""></p>
复制代码另外需要说明的是,TCP服务器线程在发送MJPEG流的时候是死循环发送的,因此TCP客户端在发送完GET指令之后,就会收到TCP服务器循环发送的图像缓存,TCP客户端会陷入忙等待状态无法再对外发送任何GET或者POST指令,从客户端使用者角度来看的效果就是网页一直在等待。 
" x$ f) S" Q' f2 F) v1 K8 e
/ ~0 E8 N8 c0 p) R二、UDP上位机UDP发送操作,同样需要先建立UDP Socket:
7 M: M/ W3 i; E O- int UDP_Send_Found(socklen_t* socket_found , struct sockaddr_in *addr , char* ip , int port)
, w" G; w! x- O# i - {
' @. H g5 v v9 D+ A3 x6 t/ P - *socket_found = socket(AF_INET, SOCK_DGRAM, 0);
# ^2 t# k* J) E; [& X( ^! G - if(*socket_found == (~0))
8 T& c ~9 T X' k' ~% K7 h - {
0 o8 I3 `8 R) y - printf("Create udp send socket failed!\n");8 v5 `; [) l. a" i3 T
- return -1;
; V9 z4 a b8 ]' M! q - }
) v4 m, R, d" c% W7 |' q. p9 P - addr->sin_family = AF_INET;
9 E$ _* e6 ?5 Y" R, R - addr->sin_addr.s_addr = inet_addr(ip);
7 F4 t0 ^4 f3 d- a - addr->sin_port = htons(port);
$ }8 n9 P% H' P' C" [* i" a - memset(addr->sin_zero, 0, 8);
" c. p3 S6 k- d- f u' j2 f( c/ [ - return 0;! v) A \) T' d
- }
复制代码
/ y7 V+ B7 p- S* a- i1 ~- f) b+ { t6 E( C
而UDP文件发送则要比HTTP发送简单得多,只需要将文件切片,每一片为固定长度的UDP帧长度,逐帧发送即可:
) @: d9 p8 _; n* D4 H1 Q( R p; c1 u% `
" C( v, W; x6 K* g
- while(fend > 0)! E! L) Q9 E1 z# l9 `+ ]: a
- {
1 F% n2 b3 T* Z7 f! @3 K6 t - memset(picture.data , 0 , sizeof(picture.data));: b; a D! F+ B: D
- fread(picture.data , UDP_FRAME_LEN , 1, fp);
, |5 v3 _+ i E. _ - if(fend >= UDP_FRAME_LEN)
% P( s8 |% G" |7 E* b5 ]0 K - {
1 S0 Y. d0 \8 V3 q - picture.length = UDP_FRAME_LEN;- Y7 Q0 f. c) j9 P; s
- picture.fin = 0;
3 H1 D+ u* e' i' a* p - }
5 f: @! y- [9 U - else
. i$ \- j q8 R% K7 O& ]! U7 P$ u4 L - {
5 P8 o! p" H- z0 t1 N8 v - picture.length = fend;+ y3 G/ ^6 y+ r4 W
- picture.fin = 1;
* i- V5 E6 q. @7 T* m% Q - }
+ x2 _ Q$ B# |$ _ - //printf("sendbytes = %d \n",sendbytes);
" W& w# I9 a. b5 S- T - sendbytes = sendto(socket_send, (char *)&picture, sizeof(struct Package), 0, (struct sockaddr*)&addr,addr_len);
& R( l! u; a( c4 F' E$ v6 b - if(sendbytes == -1)8 M# E: a$ e8 I$ Q: |. o/ p; h( [
- {
. I# j6 k) j, M" M - printf("Send Picture Failed!d\n");8 v7 `8 ^9 c3 c9 U; a4 I7 v' f
- return -1;
* ?5 R2 c. _/ y( Q9 C! A* L# ?1 m4 j/ t& \ - }
$ P% w: r6 i ?% x3 A. e ^' O - else9 y& l, ?& b P8 P
- {
+ D% F+ J! p% h N/ b2 Y$ I: q. A - fend -= UDP_FRAME_LEN;) `; W8 E5 u# G+ f
- }
( b6 Y. R$ L+ w/ g - }
复制代码
; g: c* F; X" }' J! ]: b( x' ~3 ~3 u5 K1 h1 z6 H d

8 q. L+ b; U+ P3 \2 N3 z7 O2 d, A# B* k* W$ d2 D: F) \- q O
iMX8MPlus 核心板: https://www.forlinx.com/product/136.html |