本帖最后由 飞凌-marketing 于 2022-1-27 11:00 编辑 3 P& {# E" I0 a3 q7 q. K
: k& R- I9 d( ]% g$ K1 Q
 " j6 M6 c; F j/ d) A3 W$ A
作者|donatello1996 来源 | 电子发烧友 题图|飞凌嵌入式 iMX8MPlus 核心板: https://www.forlinx.com/product/136.html, y2 C( ~- P7 Q1 |
! R7 ]3 g6 H3 r( ]& y( C& X5 z本文采用的硬件板卡为飞凌嵌入式OKMX8MP-C开发板,系统版本Linux5.4.70+Qt5.15.0,主要介绍基于HTTP网页服务器和UDP上位机的MJPG码流传输。 MJPG格式作为一种持续传输的视频码流,在远程监控领域中应用较广,而实现这种远程监控的第三方应用最常见的有两种:浏览器HTTP网页、UDP上位机。 ) I: p4 l8 b3 k0 H, ]- S
 ; S3 h9 w% F! }: l z) ^' M* H4 W' C
两者各有优势,对比鲜明,其中: 这两种应用各有优缺点,对于嵌入式开发者来说,两者都必须掌握。
" @4 m* @, S1 B0 w+ V- l! x; P一、HTTP网页服务器
e- V5 B- \" T( f6 E2 U# L# _7 B5 T先说下HTTP网页服务器获取MJPG码流的代码,首先是OKMX8MP-C在开发板端建立TCP服务器: - int TCP_Server_Found(socklen_t* socket_found , char* ip , int port)
& f- |; P+ ^% r* \5 A - {
9 k- x/ q( l; ] Y+ X; `6 z' N - struct sockaddr_in servaddr;
7 i0 m& @7 ~: R6 [* l - socklen_t addrsize = sizeof(struct sockaddr);
) U- c; a5 F6 X - bzero(&servaddr , sizeof(servaddr));
' k/ p9 J, t0 L& [! u; y - servaddr.sin_family = AF_INET;7 g6 T) {" p( q2 z- {) X
- servaddr.sin_addr.s_addr = inet_addr(ip);4 ?2 @5 H* R0 C4 U# \6 b
- servaddr.sin_port = htons(port);% y9 U" X6 v3 q# S; {
- int ret;, O# E! @+ s. ^7 b. I
- IF( (*socket_found = socket(AF_INET , SOCK_STREAM , 0)) == -1)* J' Z* d1 Y5 J I/ J% h
- {. |" d1 t, Y+ m* Q' \/ q
- printf("Create socket error: %s (errno :%d)\n",strerror(errno),errno);
( w4 z! ]6 M$ ^6 W& j [! Y/ a' v3 D - return -1;9 A/ H& Q* h) i) F) O, e
- }; P. u2 Y+ d4 e. W
- int on = 1;# S# N8 W1 Q$ B. R+ I8 J# R6 J
- if(setsockopt(*socket_found , SOL_SOCKET, SO_REUSEADDR, &on, sizeof(on)) < 0)0 h9 \1 c4 x& I& }, Q
- {
! a* K* K# E& L* p2 M+ { - printf("setsockopt error\n");
( q3 }' }* r; W6 l3 x - }8 V: ~ w& g, D6 V$ S, n- s% ]
- ret = bind(*socket_found , (struct sockaddr *)&servaddr , addrsize);
) M1 K9 I8 w( K2 e! C - if(ret == -1)
g) k5 ]9 F- p6 e6 b' M - {4 c, H- H, h" b' J' m; X
- printf("Tcp bind faiLED!\n");+ W6 Z+ F! n& G
- return -1;+ g- [" p- z7 W! F
- }* z2 e/ W: U. }% L! _5 Y
- if(listen(*socket_found , 5) == -1)5 f; ^5 y" f# S5 b% r1 O
- {
. ]4 J. e) N: v1 ]3 a7 \" \ - printf("Listen failed!\n");9 E' w' G* p& r# m. R/ ?5 g
- return -1;
$ h) h3 I9 R) G9 Y* P+ D7 o0 S - }
6 |( H9 Y0 ~ w O* g2 z6 G - return 0;# }, W) r. x1 B' U G! j! A9 K: s% |
- }
复制代码其中setsockopt()函数是可选的,一般只用于规避socket()函数的建立错误。 建立了TCP服务器后,返回的socklen_t型实参在后面的HTTP网页服务器中需要用到。 HTTP网页服务器所属的TCP操作是需要另起轮询线程来让客户端进行accept()握手操作的,accept()之前的listen()倒是只需要执行一次即可,accept()握手操作和recv()接收操作需要创建一个死循环线程: - pthread_create(&tid_tcp_web_recv , NULL , Thread_TCP_Web_Recv , NULL);
. |5 |1 h8 C! P: D' ^3 t - void * Thread_TCP_Web_Recv(void *arg)! d4 \' n* V2 s
- {7 Z3 a7 j6 ], W. o9 B4 b$ Y7 j
- 。。。* `8 E! A: c6 W& q7 ~, ?: s% @
- while(1)
3 T- R/ Z; T+ V3 z( [ - {0 U; ?, R( }( k' I6 I, r
- fd_socket_conn = accept(socket_web_server , (struct sockaddr *)&sockaddr_in_conn , &addrsize);
$ w; s' B3 X1 [- I% T+ R" Y, a1 \ - printf("fd_socket_conn = accept()\n");* m6 y; N' Z4 L7 R) x' E
- 。。。2 m3 O; Y/ G5 i4 D- s6 g8 N
- recv(fd_socket_conn , recvbuf , 1000 , 0);' o5 C; n8 t X k; K, a8 w" T
- }
' e: ^2 n' [" C1 o' d( v8 X - 。。。
' f1 L! P$ R. l! y. U. C - }
复制代码MJPG帧可以使用Grab操作获取,获取到的MJPG帧需要在TCP线程中读,在Grab操作线程中写,这种被多个线程访问的资源需要加锁防止读写冲突,即资源被Grab操作写入时,需要上锁,不允许其它线程访问,操作完成时需要解锁,允许其它线程访问: - pthread_mutex_lock(&pmt);
0 F9 @+ h) w& s7 z; s# k) b - pic_tmpbuffer = pic.tmpbuffer;7 U3 O3 |, K) {7 B$ c! L
- pic.tmpbytesused = buff.bytesused;
; B6 u1 T0 f6 |" D! g# H8 G - pic_tmpbytesused = pic.tmpbytesused;
: g4 O; s" z1 R6 f8 y - pthread_cond_broadcast(&pct);& s3 R- @% G9 z; o: A8 J
- pthread_mutex_unlock(&pmt);
复制代码线程互斥锁使用之前需要初始化: - pthread_mutex_t pmt;
: F" w4 D% A. O! U: h( K T' T - pthread_cond_t pct;' \9 |: b/ r/ a' H8 m0 g1 E1 F
- int main(int argc, char* argv[])' w4 b) o# Y$ a3 o h1 F
- {
( N* d0 ^9 |6 C% v' B( E# N( s - ...9 A. Q/ Y8 g8 Q+ v9 A- H" K3 W
- TCP_Server_Found(&socket_web_server , (char*)argv[2] , PORT_TCP);
8 ?* H5 d4 L# ^) M - pthread_mutex_init(&pmt , NULL);: q# U2 d5 u8 R8 U
- pthread_create(&tid_tcp_web_recv , NULL , Thread_TCP_Web_Recv , NULL);
4 u! N- g' V: c) Z - pthread_create(&tid_tcp_web_send , NULL , Thread_TCP_Web_Send , NULL);
3 H1 W8 s6 d @+ C - ...
`1 G0 A; C( }5 ?3 T' F - while(1)+ \3 R' J: K8 P. {" a
- {
: u$ U: \% p! A" ? - V4l2_Grab_Mjpeg(false , MJPEG_FILE_NAME);( A# m: a# V7 q- l/ f( }
- ...% @% \& }% l4 ~4 i5 Q9 D6 O
- }
: p$ P3 C: r5 e8 y s - ...
& n- s& b' d: M+ q3 H3 | - }
复制代码然后是发送的细节,发送图片文件之前,需要先发送HTTP标准头,这个相当于给发送图片或者其它类型的流数据铺路: - <p style="box-sizing: border-box; border: 0px; vertical-align: baseline; line-height: 26px;"><span style="text-indent: 32px;">
/ E& X% N* R5 w5 P - </span></p><p style="box-sizing: border-box; border: 0px; vertical-align: baseline; line-height: 26px;"></p>
复制代码- #define STD_HEADER "Connection: close\r\n" \
- o3 m+ I2 _( ] - "Server: MJPG-Streamer/0.2\r\n" \1 m$ h0 f3 M* @" a, l$ j
- "Cache-Control: no-store, no-cache, must-revalidate, pre-check=0, post-check=0, max-age=0\r\n" \- D% l# u$ V/ l, I
- "Pragma: no-cache\r\n" \3 ~1 E( I) B* k& m& e# f
- "Expires: Mon, 3 Jan 2000 12:34:56 GMT\r\n"
2 p# o" `4 O% Q1 Z! [7 ^5 m7 z - #define BOUNDARY "boundarydonotcross"
6 a( ^) D/ D5 Z) h5 x) C( m2 y - printf("preparing header\n");
( R, p$ Q- v$ I; l) B K& Q8 b/ P8 l - sprintf(buffer, "HTTP/1.0 200 OK\r\n" \% [% M+ \3 X. w; C% f5 [
- "Access-Control-Allow-Origin: *\r\n" \+ }+ f$ l) t5 V1 V, O" \. j- W
- STD_HEADER \) @$ X4 K4 X5 P. O9 _ J p, F
- "Content-Type: multipart/x-mixed-replace;boundary=" BOUNDARY "\r\n" \
+ g# P4 N+ P* m2 C, ] - "\r\n" \5 T& U l5 @, ^9 i6 t$ c! r9 V
- "--" BOUNDARY "\r\n");
6 x+ s" |- q" N1 w$ ]* h0 v+ N- R/ b - if(write(fd, buffer, strlen(buffer)) < 0)" P" v0 X w1 Y% U- j7 @) ?# F
- {
8 _' n, o0 h2 L: j% B - free(frame);
) D. F. T2 `# |, O# i3 ] - return; N* p: h5 m/ T: S; m& T; t0 E
- }
复制代码发送完HTTP标准头之后,就需要发送内容头(Content-Type),这处的Content-Type为image/jpeg,同样,HTTP标准协议里面image支持的类型远不止jpeg一种,发送完内容头之后就是正文和boundary结尾,这样帧完整的HTTP头发送到指定的TCP GET地址,就会在浏览器中显示刚刚发送的图片: - <pre class="prettyprint lang-cpp" style="box-sizing: border-box; font-family: Monaco, Menlo, Consolas, "Courier New", monospace; font-size: 16px; white-space: pre-wrap; line-height: 1.38462; color: rgb(51, 51, 51); word-break: break-all; background-color: rgb(245, 245, 245); border: 0px; border-radius: 4px; vertical-align: baseline;"> sprintf(buffer, "Content-Type: image/jpeg\r\n" \
, \4 T* Z2 U {4 g5 z8 A$ n - "Content-Length: %d\r\n" \/ v3 P$ M+ @2 E' q- a
- "X-Timestamp: %d.%06d\r\n" \2 P' a$ s# B* ^) C
- "\r\n", frame_size, (int)timestamp.tv_sec, (int)timestamp.tv_usec);6 T, d: \6 W6 F; u& M
- printf("sending intemdiate header\n");
% @/ z1 w) c9 q+ H, s& ] - if(write(fd, buffer, strlen(buffer)) < 0)
) I( v3 {3 C! G6 ?. _4 b( S& { - break;- }: ?" o" f# H1 @0 ~/ _6 q0 B
- printf("sending frame\n");
+ c6 D, x/ I- f a - if(write(fd, frame, frame_size) < 0)/ N2 J0 p# u8 t5 y; D9 R
- break;
6 C/ _% @4 G( o. S1 h3 f3 u: I - printf("sending boundary\n");9 O& r: Y8 W( }( s& ?
- sprintf(buffer, "\r\n--" BOUNDARY "\r\n");6 N; S8 q4 J6 t' Y: @. @
- if(write(fd, buffer, strlen(buffer)) < 0)
# y6 c4 Y) x2 L0 d z# o - break;</pre><p style="box-sizing: border-box; border: 0px; font-size: 16px; vertical-align: baseline; line-height: 26px; color: rgb(51, 51, 51); font-family: " helvetica="" neue",="" helvetica,="" tahoma,="" arial,="" "microsoft="" yahei",="" "hiragino="" sans="" gb",="" "wenquanyi="" micro="" hei",="" sans-serif;"=""></p>
复制代码另外需要说明的是,TCP服务器线程在发送MJPEG流的时候是死循环发送的,因此TCP客户端在发送完GET指令之后,就会收到TCP服务器循环发送的图像缓存,TCP客户端会陷入忙等待状态无法再对外发送任何GET或者POST指令,从客户端使用者角度来看的效果就是网页一直在等待。 
8 _# ]* H! ~3 ^; z0 Y2 G
4 j4 f- o* |4 v, ]$ d; C二、UDP上位机UDP发送操作,同样需要先建立UDP Socket:4 w6 Z! l" ~5 U0 ^
- int UDP_Send_Found(socklen_t* socket_found , struct sockaddr_in *addr , char* ip , int port)
6 F7 G* d0 a! N7 ]- l - {
: g% [' Y" j& r4 j - *socket_found = socket(AF_INET, SOCK_DGRAM, 0);
: } z1 C# W2 q! E - if(*socket_found == (~0)); Q4 X# Z8 W9 ^' w* i
- {
* Y- N0 g8 X3 I4 D1 I b( k - printf("Create udp send socket failed!\n");
f7 r" n" x8 f& |8 @ - return -1;
$ v- G7 e+ [7 R6 e N* l) U! {* ? F - }% l4 f8 b9 T K& {
- addr->sin_family = AF_INET;
8 ~* Y; A$ J/ x. }% F - addr->sin_addr.s_addr = inet_addr(ip);
: x+ I$ T/ x2 d - addr->sin_port = htons(port);
! \- f8 }: A& s7 T; f - memset(addr->sin_zero, 0, 8);0 L3 X. H5 @6 Z# K+ [
- return 0;
! }+ F9 `+ [1 @( p3 \" L/ d - }
复制代码
# Z3 X" k3 v" a; X! G9 |/ S' o D& C8 ]+ h1 A1 h, B+ k. @: f
而UDP文件发送则要比HTTP发送简单得多,只需要将文件切片,每一片为固定长度的UDP帧长度,逐帧发送即可:
3 U: i5 b+ o* l! f! x W$ m! R) E, E6 u& q$ F1 [
6 n; g$ E1 S, h- while(fend > 0)$ { r( Q2 }- m7 O
- {# A# Y. K6 T0 f! N
- memset(picture.data , 0 , sizeof(picture.data));
K% k0 ~5 v, A& T) M% ~ - fread(picture.data , UDP_FRAME_LEN , 1, fp);
* R. [& Y! d% H - if(fend >= UDP_FRAME_LEN)
" D0 O" s5 G3 X: H+ I2 C& ^# d - {
0 @3 h8 i! M5 z- \ - picture.length = UDP_FRAME_LEN;0 M. I1 O4 ]! g6 d i0 y8 w* _% S
- picture.fin = 0;
9 Z& J4 h' t, D d1 o4 M - }7 Y" i; f2 N- s$ }; v6 {! U
- else
' `% Q" k& ^6 N0 ]# U- I. T - {% K5 {& j% N6 z N+ ~. {9 N
- picture.length = fend;
+ s, s# d0 R ]3 @. @ - picture.fin = 1;4 v4 x; B' Y9 ^' k$ I! H
- }% V# Z* }! L) A+ r: d7 M
- //printf("sendbytes = %d \n",sendbytes);% ]) y/ r. c8 i0 r: ^2 R# ?
- sendbytes = sendto(socket_send, (char *)&picture, sizeof(struct Package), 0, (struct sockaddr*)&addr,addr_len);1 S& N- {. [8 y$ w" }$ L
- if(sendbytes == -1)4 n Y7 u& t. y4 F
- {" i; c; P/ n1 L! k
- printf("Send Picture Failed!d\n");7 i, W# o/ p2 P* k, B7 ~
- return -1;6 z& ?; {+ x1 H9 S4 F* u; V W
- }
3 [" J \1 x3 O/ q' E - else9 O: q6 D* K" A: c/ Z7 Y
- {. n* L8 M0 n' g/ A6 S3 f% G5 Q/ s9 j
- fend -= UDP_FRAME_LEN;
( `% ?* v/ d9 K3 R. X. T$ l - }* G/ D8 i! M9 S! V- s
- }
复制代码
9 Y6 m6 a' W% f8 K9 a1 m" z$ g
* w; N1 F- v& m" G/ Z; B; m
# H3 m0 H( @' p L7 T
+ {3 u# ], M5 ciMX8MPlus 核心板: https://www.forlinx.com/product/136.html |