本帖最后由 飞凌-marketing 于 2022-1-27 11:00 编辑
h8 ^# Z+ O3 S5 H
; Y2 I7 l. A4 E h4 O ) k" S# d2 Z+ t2 |3 ~1 {
作者|donatello1996 来源 | 电子发烧友 题图|飞凌嵌入式 iMX8MPlus 核心板: https://www.forlinx.com/product/136.html
K9 w* B! O* A }- s' u4 O! D: R3 E2 Q" O2 C
本文采用的硬件板卡为飞凌嵌入式OKMX8MP-C开发板,系统版本Linux5.4.70+Qt5.15.0,主要介绍基于HTTP网页服务器和UDP上位机的MJPG码流传输。 MJPG格式作为一种持续传输的视频码流,在远程监控领域中应用较广,而实现这种远程监控的第三方应用最常见的有两种:浏览器HTTP网页、UDP上位机。
; e1 k3 J3 j9 k w9 b+ d7 {5 \( J & n, E$ |: t% m. t
两者各有优势,对比鲜明,其中: 这两种应用各有优缺点,对于嵌入式开发者来说,两者都必须掌握。 1 C' B- J6 N2 d& q
一、HTTP网页服务器: O! u0 G& Z! ~
先说下HTTP网页服务器获取MJPG码流的代码,首先是OKMX8MP-C在开发板端建立TCP服务器: - int TCP_Server_Found(socklen_t* socket_found , char* ip , int port)
) `8 |- l! \$ H, |& O6 B8 ]$ b# A - {% s/ p% ~/ W5 r, `! G8 ]3 r$ s
- struct sockaddr_in servaddr;
/ ?) X' V" Y' x - socklen_t addrsize = sizeof(struct sockaddr);
8 _+ H- s- y* u# e! ~! T9 _, H - bzero(&servaddr , sizeof(servaddr));& X6 X) b4 z0 M+ M$ }
- servaddr.sin_family = AF_INET;8 u. D+ w8 P R; U( g2 y; [
- servaddr.sin_addr.s_addr = inet_addr(ip);
' H$ ]2 m4 ^2 G2 r- k1 ~ - servaddr.sin_port = htons(port);3 ^7 g; \8 n0 {. j0 I' X0 P1 ^& b
- int ret;
3 ~7 i2 h; }7 v7 [$ G - IF( (*socket_found = socket(AF_INET , SOCK_STREAM , 0)) == -1) |* c3 h5 P" J1 J! Q& G2 R3 S
- {5 h2 j c: x" ^2 y
- printf("Create socket error: %s (errno :%d)\n",strerror(errno),errno);
/ u4 V+ H) }% g% [6 U - return -1;
7 A, D% f. {- Z; Z6 U - }( L8 U/ F9 O) e7 }; z/ p9 N
- int on = 1;
: p2 Y* v# x: o/ p( o4 J; P - if(setsockopt(*socket_found , SOL_SOCKET, SO_REUSEADDR, &on, sizeof(on)) < 0)) F% p, h3 R* q( a5 ?7 m2 n
- {
. a1 M5 ], t; z* E0 ?8 ]6 R ` - printf("setsockopt error\n");
+ u% M I" Z6 z* A: p - }+ o, g9 c; B% @% ]9 J6 y
- ret = bind(*socket_found , (struct sockaddr *)&servaddr , addrsize);
5 U$ ?( |, _' C0 Q/ n) a - if(ret == -1)
; i* w! I: M6 m# u, c - {
) D. g* F: ]1 Q! `6 C - printf("Tcp bind faiLED!\n");
; r3 \& O/ J( k/ C; E. o* A" L) T5 ? - return -1;4 j) ~9 w: ~( n- e k3 |
- }# D6 J+ e, _% M! v* X( _* w; H0 J
- if(listen(*socket_found , 5) == -1)
" t6 i `4 Y4 B% V. s* f$ h* r - {
# ]9 O* z- t1 `. R$ ]+ ]9 D) y D - printf("Listen failed!\n"); j8 A- H7 b* A) i* w
- return -1;$ C8 L: f+ L; J/ l
- }1 j3 i% \: ?3 `& g1 N
- return 0;
0 C5 [9 d1 x8 E9 k; ~. x - }
复制代码其中setsockopt()函数是可选的,一般只用于规避socket()函数的建立错误。 建立了TCP服务器后,返回的socklen_t型实参在后面的HTTP网页服务器中需要用到。 HTTP网页服务器所属的TCP操作是需要另起轮询线程来让客户端进行accept()握手操作的,accept()之前的listen()倒是只需要执行一次即可,accept()握手操作和recv()接收操作需要创建一个死循环线程: - pthread_create(&tid_tcp_web_recv , NULL , Thread_TCP_Web_Recv , NULL);5 |# f6 Q* E$ G, U
- void * Thread_TCP_Web_Recv(void *arg)
' v" g8 F$ V. t% |3 @9 U$ A/ O - {
( }% z* q3 g) J6 `2 H& E2 y5 ` s - 。。。
/ @5 W. g( b0 U9 R/ u, o8 b - while(1)
* c3 K: P* Z3 l - {2 b# R7 s6 Q% @$ Q
- fd_socket_conn = accept(socket_web_server , (struct sockaddr *)&sockaddr_in_conn , &addrsize);
1 V* ?, ~/ K* X3 p: G3 Q - printf("fd_socket_conn = accept()\n");& \6 }2 i6 a! x$ h7 X/ Z$ y
- 。。。
y: ?4 G5 {% m* R$ v, u - recv(fd_socket_conn , recvbuf , 1000 , 0);, U: L* ^& Z5 J; c/ x
- }
- Q! T2 M. A( @- \ [7 X n - 。。。
. B- n% _3 x/ f& i7 J! b - }
复制代码MJPG帧可以使用Grab操作获取,获取到的MJPG帧需要在TCP线程中读,在Grab操作线程中写,这种被多个线程访问的资源需要加锁防止读写冲突,即资源被Grab操作写入时,需要上锁,不允许其它线程访问,操作完成时需要解锁,允许其它线程访问: - pthread_mutex_lock(&pmt);
7 I1 u9 @6 U& ] - pic_tmpbuffer = pic.tmpbuffer;
& S) `4 m( ~& u8 O - pic.tmpbytesused = buff.bytesused;8 Q. {- M+ B1 f4 V
- pic_tmpbytesused = pic.tmpbytesused;% U( H5 W3 Z9 G6 W- [- A/ m
- pthread_cond_broadcast(&pct);
6 T5 L8 a% q$ @* B - pthread_mutex_unlock(&pmt);
复制代码线程互斥锁使用之前需要初始化: - pthread_mutex_t pmt;
' m8 c# D" E. ?5 M6 P - pthread_cond_t pct;9 L, D, J' y% m/ B( g7 r
- int main(int argc, char* argv[])
5 Q9 i2 k! {+ {' a3 O6 X - {
, C8 e# r& ~- s$ X i - ...
# F5 }5 R: f- _0 ~- D - TCP_Server_Found(&socket_web_server , (char*)argv[2] , PORT_TCP);! O" P8 j: V+ F3 Q; A
- pthread_mutex_init(&pmt , NULL);1 W9 P0 W0 e* W5 l7 L
- pthread_create(&tid_tcp_web_recv , NULL , Thread_TCP_Web_Recv , NULL);
) ]7 s; G- q6 B3 c - pthread_create(&tid_tcp_web_send , NULL , Thread_TCP_Web_Send , NULL);
8 j* H U4 t0 {+ z" X% V( S# C" W - ...
K" ^5 G9 c! T( _4 b0 w0 C - while(1)* n: V6 l2 {8 a+ E* q0 `5 G; f
- {5 f, |. H, ~. [9 b2 j4 J
- V4l2_Grab_Mjpeg(false , MJPEG_FILE_NAME);
* y# r# T# u% I3 g( l$ L - ...
! ]) p) W( r; [( M3 O; p - }6 X: `2 J& D d& Q8 I! R; x7 t$ F
- ... c8 P% s; }- t. b4 T3 j
- }
复制代码然后是发送的细节,发送图片文件之前,需要先发送HTTP标准头,这个相当于给发送图片或者其它类型的流数据铺路: - <p style="box-sizing: border-box; border: 0px; vertical-align: baseline; line-height: 26px;"><span style="text-indent: 32px;">
1 h, n8 H5 \1 H& {8 y: u% `& n3 L- D - </span></p><p style="box-sizing: border-box; border: 0px; vertical-align: baseline; line-height: 26px;"></p>
复制代码- #define STD_HEADER "Connection: close\r\n" \" H9 E( ?4 ?: U \' y
- "Server: MJPG-Streamer/0.2\r\n" \5 Y, m F7 F5 N. w
- "Cache-Control: no-store, no-cache, must-revalidate, pre-check=0, post-check=0, max-age=0\r\n" \
- M$ D8 F$ X4 C5 d' k - "Pragma: no-cache\r\n" \# ^1 L: d. {3 g g% F
- "Expires: Mon, 3 Jan 2000 12:34:56 GMT\r\n"7 l U' R. Q8 y ~3 R
- #define BOUNDARY "boundarydonotcross"
7 K9 D$ ^3 I8 q* } - printf("preparing header\n");
( d) d$ O/ N4 f: H - sprintf(buffer, "HTTP/1.0 200 OK\r\n" \
4 Q$ X' ~+ u* v( g - "Access-Control-Allow-Origin: *\r\n" \1 T$ r% x- D6 ]. O; _1 n
- STD_HEADER \
' p T' ]2 D# p - "Content-Type: multipart/x-mixed-replace;boundary=" BOUNDARY "\r\n" \+ {# U, p9 i$ k0 b2 D: n- q
- "\r\n" \
4 ~4 ]& H) [2 t8 j# G - "--" BOUNDARY "\r\n");3 J5 E% e, H% R# c! t- D# O
- if(write(fd, buffer, strlen(buffer)) < 0)
, i3 f, M# U+ d% P; k3 V - {/ o. p7 R1 i4 }, j. `" C3 f
- free(frame); D: f; \( v4 P3 L
- return;
( W2 l# T$ g8 \8 o$ h3 } - }
复制代码发送完HTTP标准头之后,就需要发送内容头(Content-Type),这处的Content-Type为image/jpeg,同样,HTTP标准协议里面image支持的类型远不止jpeg一种,发送完内容头之后就是正文和boundary结尾,这样帧完整的HTTP头发送到指定的TCP GET地址,就会在浏览器中显示刚刚发送的图片: - <pre class="prettyprint lang-cpp" style="box-sizing: border-box; font-family: Monaco, Menlo, Consolas, "Courier New", monospace; font-size: 16px; white-space: pre-wrap; line-height: 1.38462; color: rgb(51, 51, 51); word-break: break-all; background-color: rgb(245, 245, 245); border: 0px; border-radius: 4px; vertical-align: baseline;"> sprintf(buffer, "Content-Type: image/jpeg\r\n" \* B1 T3 S ^, m# s6 Y6 H
- "Content-Length: %d\r\n" \" F5 b/ r& c2 I- F j0 B3 w9 v
- "X-Timestamp: %d.%06d\r\n" \
. v) S' J) q+ \ - "\r\n", frame_size, (int)timestamp.tv_sec, (int)timestamp.tv_usec); b6 s# o6 m# k9 m, R* H
- printf("sending intemdiate header\n");3 b7 L. z& A& a1 S
- if(write(fd, buffer, strlen(buffer)) < 0)
0 u3 b* w1 u; ^4 D& G - break;
8 g+ I" j; R6 e - printf("sending frame\n");$ v5 @* e' P, |# B& \
- if(write(fd, frame, frame_size) < 0)1 R4 f8 w6 Z( P6 y5 M$ V3 Q5 R
- break;
. F* H: q) d: z# D - printf("sending boundary\n");
" e, N* l* i% v# r) `$ P3 ]% m - sprintf(buffer, "\r\n--" BOUNDARY "\r\n");; d, o6 n- f6 E% m1 _
- if(write(fd, buffer, strlen(buffer)) < 0)8 ]- a0 Y2 G( ]4 z
- break;</pre><p style="box-sizing: border-box; border: 0px; font-size: 16px; vertical-align: baseline; line-height: 26px; color: rgb(51, 51, 51); font-family: " helvetica="" neue",="" helvetica,="" tahoma,="" arial,="" "microsoft="" yahei",="" "hiragino="" sans="" gb",="" "wenquanyi="" micro="" hei",="" sans-serif;"=""></p>
复制代码另外需要说明的是,TCP服务器线程在发送MJPEG流的时候是死循环发送的,因此TCP客户端在发送完GET指令之后,就会收到TCP服务器循环发送的图像缓存,TCP客户端会陷入忙等待状态无法再对外发送任何GET或者POST指令,从客户端使用者角度来看的效果就是网页一直在等待。 
& }/ G- p- b4 P4 F3 Y; p" v* N
2 B/ Q9 l& h* O" F1 O6 H二、UDP上位机UDP发送操作,同样需要先建立UDP Socket:/ V9 t# Z# w. V+ k) \( E7 A
- int UDP_Send_Found(socklen_t* socket_found , struct sockaddr_in *addr , char* ip , int port)0 x* D4 k5 {$ L( H
- {8 c; N d \+ k/ {& C2 C7 W
- *socket_found = socket(AF_INET, SOCK_DGRAM, 0);
- N& U- x: w4 R - if(*socket_found == (~0))
0 M8 }( A" y4 x _: R x3 d - {3 Z1 c- {' a! u: Q. B/ ]
- printf("Create udp send socket failed!\n");
& w3 S2 D- K/ f3 E/ }4 a2 J - return -1;
5 d; q& g1 `# s7 n" O9 W - }
8 s/ Z; {% ^( u1 o, {! d& N - addr->sin_family = AF_INET;6 k, h$ i& w; @+ x; Z" y0 {, p1 B& V
- addr->sin_addr.s_addr = inet_addr(ip);2 U. v( H0 Z+ N. t4 Z' u3 e" t" R9 H
- addr->sin_port = htons(port);( L) Y9 U7 h5 ?6 J) C
- memset(addr->sin_zero, 0, 8);
$ y5 e! B, Y. ` - return 0;
+ W" y) L9 L- Q+ m! l - }
复制代码 ' ]! E+ |! c6 E
9 x# R/ s3 E0 c& b5 ?而UDP文件发送则要比HTTP发送简单得多,只需要将文件切片,每一片为固定长度的UDP帧长度,逐帧发送即可:
. a5 z4 {$ Z5 R; s9 s: \: J8 @" H5 h$ e9 x: M* n9 Z1 t0 L! d1 k
7 I$ r' E- G! x! o# \2 N( H: {5 F3 E- while(fend > 0)
6 ^& ^$ t) s. S+ N2 |9 t3 K - {
% R5 A0 G [5 v: L6 Z6 W6 u6 D - memset(picture.data , 0 , sizeof(picture.data));5 u9 {* J3 }0 l* t5 T- o K- N L
- fread(picture.data , UDP_FRAME_LEN , 1, fp);
# H1 X, n2 {9 |! c - if(fend >= UDP_FRAME_LEN)
4 H" V: P2 Z. C2 ~9 p) s - {
3 w) v( t9 W6 N. b) @6 Q - picture.length = UDP_FRAME_LEN;
( Z2 T5 G, q. e# { - picture.fin = 0;
2 C3 }! g5 ~' e" n: n' C6 n) ^ - }
/ _: ~. |: M. w - else
8 Y- R. v! a- z+ J- n# t - {' _: q3 g1 m& Z3 G
- picture.length = fend;
! g7 {. @! a" p# e$ e3 r - picture.fin = 1;9 ?1 T$ L! K- G2 x+ A3 Q
- }% n' r1 A( u1 [' Y! A1 O- _6 r
- //printf("sendbytes = %d \n",sendbytes);* \0 P6 _) a+ M6 B6 t( ^
- sendbytes = sendto(socket_send, (char *)&picture, sizeof(struct Package), 0, (struct sockaddr*)&addr,addr_len);
' O6 F9 K8 O4 x - if(sendbytes == -1)& [ k' D; Z1 Z
- {7 D4 R2 T7 k% f) P4 ~8 Y1 H; S& R2 I
- printf("Send Picture Failed!d\n");
q5 W+ u# @* o) |1 p - return -1;7 s; _. B. @. |7 v# r& y* p
- }
1 ?' a8 X2 L% }( ]. y( p4 g9 @ - else- k% J' X1 Y, a; f3 G, [
- {6 L+ Y; q6 W0 E S
- fend -= UDP_FRAME_LEN;9 x& H0 a$ C$ w( `0 D' \
- }! z+ J) s) A! R: {# r9 f5 W+ y4 m
- }
复制代码
0 [, R( U: N \. v# C+ U
9 J1 R2 L. J2 z* O& l5 p$ I, Q
4 H) H n3 P9 T" u# Y! F2 \& X1 w; u; N9 c) ~: ^
iMX8MPlus 核心板: https://www.forlinx.com/product/136.html |