本帖最后由 飞凌-marketing 于 2022-1-27 11:00 编辑
8 n( |# e a7 t, Z/ d( ?/ I$ y' P3 I3 Y4 X
 5 p4 u! F* w7 N) ]7 U8 S) f
作者|donatello1996 来源 | 电子发烧友 题图|飞凌嵌入式 iMX8MPlus 核心板: https://www.forlinx.com/product/136.html
! X7 V5 b' q0 h" m. `
+ K9 C5 t. R L本文采用的硬件板卡为飞凌嵌入式OKMX8MP-C开发板,系统版本Linux5.4.70+Qt5.15.0,主要介绍基于HTTP网页服务器和UDP上位机的MJPG码流传输。 MJPG格式作为一种持续传输的视频码流,在远程监控领域中应用较广,而实现这种远程监控的第三方应用最常见的有两种:浏览器HTTP网页、UDP上位机。 . L, [8 q n" |. T8 u$ l4 C

z6 J9 _8 d u( {+ D/ J; A6 X两者各有优势,对比鲜明,其中: 这两种应用各有优缺点,对于嵌入式开发者来说,两者都必须掌握。 ! e# B9 d0 N% A, i5 z6 z
一、HTTP网页服务器
+ x; p4 B. q- r. |先说下HTTP网页服务器获取MJPG码流的代码,首先是OKMX8MP-C在开发板端建立TCP服务器: - int TCP_Server_Found(socklen_t* socket_found , char* ip , int port): `4 p8 C& K. e
- {/ s0 F3 ]: |* Z6 x0 F
- struct sockaddr_in servaddr;7 |) _! f6 h1 Q+ n
- socklen_t addrsize = sizeof(struct sockaddr);
) C" T( X$ ~) L3 g O, v - bzero(&servaddr , sizeof(servaddr));/ f9 k! Z9 B4 h0 k; U
- servaddr.sin_family = AF_INET;% _9 f+ O' B) H$ a
- servaddr.sin_addr.s_addr = inet_addr(ip);
% ^; s+ }: M8 A+ w) j% K - servaddr.sin_port = htons(port);
1 r, U& v- w" J% V$ t# f - int ret;, l9 v; R5 L: b
- IF( (*socket_found = socket(AF_INET , SOCK_STREAM , 0)) == -1)
/ E' h; O7 {0 x' I' |# n - {" \+ |+ @ `1 `9 X
- printf("Create socket error: %s (errno :%d)\n",strerror(errno),errno);* c+ X+ Q1 e* _( y
- return -1;$ ]4 `( ^& O/ a# M: L
- }
* v% ]( O; H% `& ?6 i) r# g - int on = 1;( G$ Q) S8 c/ t) t- ^" V
- if(setsockopt(*socket_found , SOL_SOCKET, SO_REUSEADDR, &on, sizeof(on)) < 0)* ]% x% ?: V. k k; ?" t: [
- {2 R, q! w. c- T0 p3 V/ x
- printf("setsockopt error\n");
) S0 q9 \; r! h - }2 ^) d2 f$ e* A. @) g
- ret = bind(*socket_found , (struct sockaddr *)&servaddr , addrsize);
0 I8 g! E5 K0 A2 t' e - if(ret == -1)% ^4 |7 w2 P( X5 t9 O- R- Z
- { L' ~. a- ~/ `' b0 F8 s
- printf("Tcp bind faiLED!\n");
% I+ f4 S$ h$ ? - return -1;
: Y# ?! j8 Y% _2 S0 l( V! y; i - }$ s# x' X8 {8 v
- if(listen(*socket_found , 5) == -1). }4 M# b0 {/ s9 _! s1 Z8 T) e5 w
- {4 n' P/ h9 N3 B- s7 R
- printf("Listen failed!\n");
: M$ `. P V" P2 P# n# X - return -1;
- Q ?' X2 a% V# \' s - }
6 L& x6 r: K4 x, N" y& Q! ~ - return 0;8 M0 r0 q7 O# N( v/ o
- }
复制代码其中setsockopt()函数是可选的,一般只用于规避socket()函数的建立错误。 建立了TCP服务器后,返回的socklen_t型实参在后面的HTTP网页服务器中需要用到。 HTTP网页服务器所属的TCP操作是需要另起轮询线程来让客户端进行accept()握手操作的,accept()之前的listen()倒是只需要执行一次即可,accept()握手操作和recv()接收操作需要创建一个死循环线程: - pthread_create(&tid_tcp_web_recv , NULL , Thread_TCP_Web_Recv , NULL);( k- X( G! N) ~6 q
- void * Thread_TCP_Web_Recv(void *arg)' N" s' H6 r* T
- {9 m# [# m1 i5 v5 B9 n' y* h6 D
- 。。。0 s \% Q/ O, z1 v5 j
- while(1)
) V4 Z2 [$ k+ l k" x4 O7 H2 S, ^ - {
6 H0 r5 \2 K: P; T' W" I - fd_socket_conn = accept(socket_web_server , (struct sockaddr *)&sockaddr_in_conn , &addrsize);
5 p: }& s5 ^: Z. [. v - printf("fd_socket_conn = accept()\n");) u/ c2 N& I: _( f% B* H% @' C
- 。。。" ?; \( C' v X, F! T# K* v
- recv(fd_socket_conn , recvbuf , 1000 , 0);
5 l& o8 z/ @! x. J# U - }
" \; K5 }, u: t" ?; D- f2 g - 。。。% z f8 v' p: [* E! Z
- }
复制代码MJPG帧可以使用Grab操作获取,获取到的MJPG帧需要在TCP线程中读,在Grab操作线程中写,这种被多个线程访问的资源需要加锁防止读写冲突,即资源被Grab操作写入时,需要上锁,不允许其它线程访问,操作完成时需要解锁,允许其它线程访问: - pthread_mutex_lock(&pmt);& H, }7 J: f& \. ~0 m: v0 V
- pic_tmpbuffer = pic.tmpbuffer;
3 \! S( V! w7 c n$ W - pic.tmpbytesused = buff.bytesused;
% U8 l: ^" F9 T - pic_tmpbytesused = pic.tmpbytesused;
0 ^; D7 j+ ` r/ L# H; [ - pthread_cond_broadcast(&pct);6 s. T) ~8 L. d. _9 ~
- pthread_mutex_unlock(&pmt);
复制代码线程互斥锁使用之前需要初始化: - pthread_mutex_t pmt;5 Y4 E* ]0 k. B! D% M
- pthread_cond_t pct;/ R3 K; I+ v. g
- int main(int argc, char* argv[])5 [( d* N6 k: a
- {
* D% Q: H* s B6 F0 W# Q - ...
" H$ [( x- N) ]& A - TCP_Server_Found(&socket_web_server , (char*)argv[2] , PORT_TCP);( I( |! s! ^) s. D0 i. n+ r
- pthread_mutex_init(&pmt , NULL);
' z" b( v( q4 [: Z( Z5 M7 [ - pthread_create(&tid_tcp_web_recv , NULL , Thread_TCP_Web_Recv , NULL);
9 H1 Q6 ~. l5 K* B5 N9 x" S+ y - pthread_create(&tid_tcp_web_send , NULL , Thread_TCP_Web_Send , NULL);. J3 d9 s7 C# A) H3 z2 s$ W
- ..., ~) k W/ c5 K
- while(1)( ~% a+ E9 }2 c$ o2 C; l
- {
% V* F t% g# H6 N3 g3 } - V4l2_Grab_Mjpeg(false , MJPEG_FILE_NAME);7 u* v) ~6 S) {* E4 y
- ...
* ^ E: r3 B1 e; d# V5 d" l - }, J0 e5 t: b9 b! k. l c' a
- ..., m8 e2 w( T }& n- d
- }
复制代码然后是发送的细节,发送图片文件之前,需要先发送HTTP标准头,这个相当于给发送图片或者其它类型的流数据铺路: - <p style="box-sizing: border-box; border: 0px; vertical-align: baseline; line-height: 26px;"><span style="text-indent: 32px;">
8 H1 f- Y5 e/ ?6 l. V, b - </span></p><p style="box-sizing: border-box; border: 0px; vertical-align: baseline; line-height: 26px;"></p>
复制代码- #define STD_HEADER "Connection: close\r\n" \* s% {2 ~5 \" l$ X6 |
- "Server: MJPG-Streamer/0.2\r\n" \+ I. p- @2 J* S2 a- z! A5 N
- "Cache-Control: no-store, no-cache, must-revalidate, pre-check=0, post-check=0, max-age=0\r\n" \6 s4 b2 B- D8 G5 T. N
- "Pragma: no-cache\r\n" \$ V4 c4 }7 k2 O) H) G
- "Expires: Mon, 3 Jan 2000 12:34:56 GMT\r\n"5 a. ^/ k4 b, k" v8 }1 A+ g, q
- #define BOUNDARY "boundarydonotcross", ^1 J' m( D# q0 r% Y% j
- printf("preparing header\n");
1 N3 l; S* [/ f' Z- e9 w. H - sprintf(buffer, "HTTP/1.0 200 OK\r\n" \
8 N+ P3 g6 H1 G - "Access-Control-Allow-Origin: *\r\n" \' t+ f2 V4 }1 b& x
- STD_HEADER \8 z- |4 A5 ]" c% k/ g: W# d3 Y x# `
- "Content-Type: multipart/x-mixed-replace;boundary=" BOUNDARY "\r\n" \
1 I! X7 n) d0 B% v& W2 w - "\r\n" \
- H/ M% T: f$ D( {, x* J - "--" BOUNDARY "\r\n");
6 V- r9 h4 j Q* } - if(write(fd, buffer, strlen(buffer)) < 0)& p( m* y" z8 D! k# \
- {
, Z+ e( ]* Z" m% t( L" k9 Y+ v+ ^ - free(frame);
2 Y ^0 j$ T* I8 ?3 q - return;7 v7 G' D' V5 v' Q3 U, J/ z
- }
复制代码发送完HTTP标准头之后,就需要发送内容头(Content-Type),这处的Content-Type为image/jpeg,同样,HTTP标准协议里面image支持的类型远不止jpeg一种,发送完内容头之后就是正文和boundary结尾,这样帧完整的HTTP头发送到指定的TCP GET地址,就会在浏览器中显示刚刚发送的图片: - <pre class="prettyprint lang-cpp" style="box-sizing: border-box; font-family: Monaco, Menlo, Consolas, "Courier New", monospace; font-size: 16px; white-space: pre-wrap; line-height: 1.38462; color: rgb(51, 51, 51); word-break: break-all; background-color: rgb(245, 245, 245); border: 0px; border-radius: 4px; vertical-align: baseline;"> sprintf(buffer, "Content-Type: image/jpeg\r\n" \
& ^; y7 n o( y H7 p# H8 ] - "Content-Length: %d\r\n" \0 ]1 e# O2 u! l% g
- "X-Timestamp: %d.%06d\r\n" \
3 s. V& t) p* c - "\r\n", frame_size, (int)timestamp.tv_sec, (int)timestamp.tv_usec);9 T. [8 u: e6 v; ^, R
- printf("sending intemdiate header\n");# }# }6 A6 L, y' a/ H
- if(write(fd, buffer, strlen(buffer)) < 0): S! ~# K4 F ]0 b2 I
- break;
1 a T" C! a# h+ E - printf("sending frame\n");
, z1 f% f4 m. T d$ q2 |8 S - if(write(fd, frame, frame_size) < 0)
* C: B/ {: N5 O' z5 S: Q1 L9 a- d+ T - break;
s" F* I$ [% \; H( N, w - printf("sending boundary\n");
, Q) a! F# l+ o" v* |( C) n - sprintf(buffer, "\r\n--" BOUNDARY "\r\n");
. _5 d7 r- s3 {* R3 K - if(write(fd, buffer, strlen(buffer)) < 0)$ p) q/ y1 w6 X- A
- break;</pre><p style="box-sizing: border-box; border: 0px; font-size: 16px; vertical-align: baseline; line-height: 26px; color: rgb(51, 51, 51); font-family: " helvetica="" neue",="" helvetica,="" tahoma,="" arial,="" "microsoft="" yahei",="" "hiragino="" sans="" gb",="" "wenquanyi="" micro="" hei",="" sans-serif;"=""></p>
复制代码另外需要说明的是,TCP服务器线程在发送MJPEG流的时候是死循环发送的,因此TCP客户端在发送完GET指令之后,就会收到TCP服务器循环发送的图像缓存,TCP客户端会陷入忙等待状态无法再对外发送任何GET或者POST指令,从客户端使用者角度来看的效果就是网页一直在等待。  ) L/ s2 E9 m. U! V3 Q$ c

! @/ p$ [+ d# [6 Y8 w二、UDP上位机UDP发送操作,同样需要先建立UDP Socket:# ]" {0 R$ [' K* `" ?! c
- int UDP_Send_Found(socklen_t* socket_found , struct sockaddr_in *addr , char* ip , int port) l7 i9 P' z/ Y; m' \
- {
9 ?; {6 S- ~! N( Q0 D+ J" r/ l+ @ - *socket_found = socket(AF_INET, SOCK_DGRAM, 0);
1 V' I, P% V: O+ b% |% q. |! {) A5 q4 @ - if(*socket_found == (~0))6 F+ J. G5 p( x- E5 ?
- {" s1 u9 A7 J8 y
- printf("Create udp send socket failed!\n");
- N5 T) y' h# ?2 f3 Q3 ^5 i - return -1;! @( U- P# T7 P' P% u* w# ~
- }
, w! f4 n) x% U% p- k - addr->sin_family = AF_INET;
- R8 [ w _) p9 n; G, Y( | - addr->sin_addr.s_addr = inet_addr(ip);: q# e- y z/ m4 u/ \ Y& X
- addr->sin_port = htons(port);
, a+ I) M! D/ t) R! N - memset(addr->sin_zero, 0, 8);0 z; j0 S" D6 h" V# G" a# H
- return 0;4 I. b R; W& ]8 J
- }
复制代码
* G; w5 U' c1 }( a
, X1 k/ P9 X& _' Y) w6 R9 m而UDP文件发送则要比HTTP发送简单得多,只需要将文件切片,每一片为固定长度的UDP帧长度,逐帧发送即可:
3 w$ I- w8 m# \* y9 W: G1 |! @3 j$ |. w/ F
( J! U- g$ `: E; z( P6 m+ k* g, G- while(fend > 0)$ E$ k: E5 g* p; C9 h3 m K* K
- {5 u; n% [! j/ P# [# \
- memset(picture.data , 0 , sizeof(picture.data));
. Z ~4 Z. i, o - fread(picture.data , UDP_FRAME_LEN , 1, fp);# _( a' ?/ o3 |+ p
- if(fend >= UDP_FRAME_LEN)3 X- g u( ~ ?, V" c) N7 J% L
- {
+ `9 a8 b7 c4 x. p' [1 G; `3 q - picture.length = UDP_FRAME_LEN;
# _8 Y3 L M- R2 E; d7 v - picture.fin = 0;
% C3 b6 U% P7 q/ g7 n0 J - }
6 w7 w4 u9 O# u% _% t - else
) M$ x! q% X; k3 ~4 d) u - {: ^4 ~) N- F$ P0 ^! f5 v$ {# Q2 _" C
- picture.length = fend;) K0 z! [5 J3 \: R* `! ~* l. T4 n
- picture.fin = 1;
, z* Q: s5 L4 X& s2 d( d - }) Z% ~% R4 J8 G# r% D, S/ E
- //printf("sendbytes = %d \n",sendbytes);
* ^6 C6 l, }: L' q - sendbytes = sendto(socket_send, (char *)&picture, sizeof(struct Package), 0, (struct sockaddr*)&addr,addr_len);
/ g6 ?1 w; N$ N: d- ` - if(sendbytes == -1)$ s0 \( [4 ]$ D$ ?$ `+ W. j
- {
! v6 E0 k' f4 l9 D# o% t6 c - printf("Send Picture Failed!d\n");
- C8 n% p' [' }" o6 s% v; Q' Z$ r" ^7 B - return -1;- U8 P8 K" \& H$ `1 e
- }1 U4 A* K9 J: R7 f3 k( d" ?% r
- else1 M9 v1 r2 m! J, G4 W8 }8 g
- {, T9 P' J! E/ S8 v: o
- fend -= UDP_FRAME_LEN;/ |' ?/ l( p$ `1 E. d1 k8 r
- }. {# x1 v: b, P4 K2 M4 }% y; g
- }
复制代码
5 y& P1 `* S/ R0 `4 J+ B1 }# s- h2 N3 W/ K" y/ r4 ?2 s' m

1 i! N1 M+ C! P# h
w: C5 L) m4 }2 j. W3 n* QiMX8MPlus 核心板: https://www.forlinx.com/product/136.html |