本帖最后由 飞凌-marketing 于 2022-1-27 11:00 编辑
z' A2 q! u1 n8 U0 _# G6 S
' c/ X* \: v3 X0 h! K0 X" P6 ?
; F. z" _0 }4 _: w# G% u作者|donatello1996 来源 | 电子发烧友 题图|飞凌嵌入式 iMX8MPlus 核心板: https://www.forlinx.com/product/136.html
# |, P8 M+ G1 _* U3 _8 [- ?
7 e" h6 c& L' N. q本文采用的硬件板卡为飞凌嵌入式OKMX8MP-C开发板,系统版本Linux5.4.70+Qt5.15.0,主要介绍基于HTTP网页服务器和UDP上位机的MJPG码流传输。 MJPG格式作为一种持续传输的视频码流,在远程监控领域中应用较广,而实现这种远程监控的第三方应用最常见的有两种:浏览器HTTP网页、UDP上位机。 ( w& Z9 a1 Y8 U

: |2 U3 n% X0 `" ?, M- [5 j2 ]" c两者各有优势,对比鲜明,其中: 这两种应用各有优缺点,对于嵌入式开发者来说,两者都必须掌握。 / f3 D) K! }2 T9 K, t0 M6 p
一、HTTP网页服务器
: j9 M) y8 p" L V先说下HTTP网页服务器获取MJPG码流的代码,首先是OKMX8MP-C在开发板端建立TCP服务器: - int TCP_Server_Found(socklen_t* socket_found , char* ip , int port)
% q" f+ }* a+ c9 q' _ - {* ^' Z/ S4 b% w0 b1 O
- struct sockaddr_in servaddr;3 |$ y4 o o4 U3 R
- socklen_t addrsize = sizeof(struct sockaddr);
" I0 x, C* |9 g% g2 V+ x - bzero(&servaddr , sizeof(servaddr));
. @3 @1 x) o0 w0 t* Z - servaddr.sin_family = AF_INET;
, [, ^- T2 }4 G+ O8 n - servaddr.sin_addr.s_addr = inet_addr(ip);
' S1 F7 @$ p9 `! ~' ~% X: U - servaddr.sin_port = htons(port);' ?9 j0 L. `4 {8 z: f2 |
- int ret;; F5 r) ?+ \* g- B% A4 F" }& u- [
- IF( (*socket_found = socket(AF_INET , SOCK_STREAM , 0)) == -1)
) i$ |1 ]' I" O. @& N n - {) E; [7 Q. ?) R( p0 x; n; s9 J
- printf("Create socket error: %s (errno :%d)\n",strerror(errno),errno);
- t6 N% G" f* S5 P! G - return -1;4 m8 s! v! ?0 s0 l
- }
" M8 P# B+ v, s; \ o# c - int on = 1;
9 a9 Q4 l$ v' M- |0 ~5 E7 V - if(setsockopt(*socket_found , SOL_SOCKET, SO_REUSEADDR, &on, sizeof(on)) < 0)# A2 t' Q# P, s; v: T+ R ^; b
- {7 |0 L, M) h) F$ s; h( h
- printf("setsockopt error\n");
" j( n: ?, z8 M - }0 J' d, w/ ?' s5 E. `, p% u) O
- ret = bind(*socket_found , (struct sockaddr *)&servaddr , addrsize);% m2 a2 H7 O' M& n
- if(ret == -1)
9 V; m2 q; Z! E# x - {
; k n" N( H; ] - printf("Tcp bind faiLED!\n");! C" t/ C% {$ h0 a
- return -1;1 C6 ~, M$ ?/ _$ w! i+ ^" P
- }- g) h7 i4 s; U( L% E
- if(listen(*socket_found , 5) == -1)0 C* K8 G* U6 l% t" s% A% z' k0 u
- {
$ E* _; G* ?& _ - printf("Listen failed!\n");8 E: G- o: q, _1 Z, M* e
- return -1;& J, L% j q3 [/ O" D
- }
7 U% K, E' V9 v n; K3 p1 _ - return 0;) N/ J& g4 Z- Z, Y$ F
- }
复制代码其中setsockopt()函数是可选的,一般只用于规避socket()函数的建立错误。 建立了TCP服务器后,返回的socklen_t型实参在后面的HTTP网页服务器中需要用到。 HTTP网页服务器所属的TCP操作是需要另起轮询线程来让客户端进行accept()握手操作的,accept()之前的listen()倒是只需要执行一次即可,accept()握手操作和recv()接收操作需要创建一个死循环线程: - pthread_create(&tid_tcp_web_recv , NULL , Thread_TCP_Web_Recv , NULL);
4 z+ q$ C* M+ t ]0 ] - void * Thread_TCP_Web_Recv(void *arg)3 |6 q* W1 w0 \
- {
, j2 B8 X( H5 }* \+ k' s - 。。。
% `, [( [4 n: F2 J* G8 h$ \- ?0 g! b - while(1)
, p3 O- d* _( ]# I8 E - {
3 S6 x0 Q/ w( t$ ^6 F6 ]+ J7 v2 } - fd_socket_conn = accept(socket_web_server , (struct sockaddr *)&sockaddr_in_conn , &addrsize);; e: T; y% W+ A8 M+ A: O2 C; f
- printf("fd_socket_conn = accept()\n");' ]. C- K7 M( T" u$ j. X& v
- 。。。
2 W" M9 H% P: V - recv(fd_socket_conn , recvbuf , 1000 , 0);
. b8 @3 j" G& J- l) U - }; `$ z1 o) f8 M' @/ J, v8 K! q! a
- 。。。; ~* s# [2 y' w1 Q" Q2 ]9 Q7 g
- }
复制代码MJPG帧可以使用Grab操作获取,获取到的MJPG帧需要在TCP线程中读,在Grab操作线程中写,这种被多个线程访问的资源需要加锁防止读写冲突,即资源被Grab操作写入时,需要上锁,不允许其它线程访问,操作完成时需要解锁,允许其它线程访问: - pthread_mutex_lock(&pmt);
6 H) h1 O8 u8 m! I+ B7 b, |1 I - pic_tmpbuffer = pic.tmpbuffer;
; r8 Y$ Q8 e$ B* L, G5 C - pic.tmpbytesused = buff.bytesused;
Q+ b+ U% l8 X7 V - pic_tmpbytesused = pic.tmpbytesused;8 t/ m. s2 u W R6 d3 D
- pthread_cond_broadcast(&pct);+ v: L4 G. Z/ T8 g$ Y
- pthread_mutex_unlock(&pmt);
复制代码线程互斥锁使用之前需要初始化: - pthread_mutex_t pmt;; m2 |! L2 K; k& r! I
- pthread_cond_t pct;$ y3 R) [9 ]+ p/ H
- int main(int argc, char* argv[])
+ E1 m5 t, ]. I. } - {$ \ L* k4 w( a8 @6 C; k
- ...# u- k! G3 G+ _, v! ?! P
- TCP_Server_Found(&socket_web_server , (char*)argv[2] , PORT_TCP);! ?. n6 j! x6 K( z& G
- pthread_mutex_init(&pmt , NULL);2 g! T3 U% ~* |9 Y, Y1 q4 s
- pthread_create(&tid_tcp_web_recv , NULL , Thread_TCP_Web_Recv , NULL);
7 O* P4 H$ W3 z% B- o* y1 }% w - pthread_create(&tid_tcp_web_send , NULL , Thread_TCP_Web_Send , NULL);
$ U* Q$ g5 _4 |" I - ...
' q9 ^9 M: j* b' u) ~% O - while(1)
+ {. n/ A7 Q2 A- T3 ?; S; [% o - {; T* v1 W! r+ {9 _' o9 y
- V4l2_Grab_Mjpeg(false , MJPEG_FILE_NAME);
* ^) e( I& D8 C# f - ...
; z) y. H7 e& y/ B+ P# m2 w2 c1 C - }
7 Q( A! f8 T2 M2 ~; E* B5 u" v - ..." w! _0 h6 s$ A5 g2 \+ w2 U
- }
复制代码然后是发送的细节,发送图片文件之前,需要先发送HTTP标准头,这个相当于给发送图片或者其它类型的流数据铺路: - <p style="box-sizing: border-box; border: 0px; vertical-align: baseline; line-height: 26px;"><span style="text-indent: 32px;">
# h; G* ?, |. z& Y+ v/ p4 e* T - </span></p><p style="box-sizing: border-box; border: 0px; vertical-align: baseline; line-height: 26px;"></p>
复制代码- #define STD_HEADER "Connection: close\r\n" \
/ H; @# q! c2 } - "Server: MJPG-Streamer/0.2\r\n" \
6 x5 P+ Z9 a: X, I4 J8 s# T - "Cache-Control: no-store, no-cache, must-revalidate, pre-check=0, post-check=0, max-age=0\r\n" \
" o. Z7 o. V7 R - "Pragma: no-cache\r\n" \7 g* d4 _' O" V# y3 T
- "Expires: Mon, 3 Jan 2000 12:34:56 GMT\r\n"
. j8 w" X% ^0 E! M! N8 { - #define BOUNDARY "boundarydonotcross"3 L0 m: J% i% n. z4 \/ ]' X
- printf("preparing header\n");% ^, ~3 l5 t R2 X/ N0 j) s
- sprintf(buffer, "HTTP/1.0 200 OK\r\n" \
" G& w9 W l; F% R. g/ A e. V - "Access-Control-Allow-Origin: *\r\n" \
; H) Z2 b6 x' G5 u - STD_HEADER \
' ^( ~' q3 j8 @( A; \ - "Content-Type: multipart/x-mixed-replace;boundary=" BOUNDARY "\r\n" \
3 a3 j: u: p- X! p' d3 S - "\r\n" \
* s2 O/ ~. K; I4 o b - "--" BOUNDARY "\r\n");6 @. G/ w$ n! a
- if(write(fd, buffer, strlen(buffer)) < 0)
) a0 e: L, h! j3 q& P! I - {
; a V U5 F5 `8 S" v* P6 ?5 T2 w u - free(frame);
% ~$ ?4 s# g6 k9 w& A - return;
0 L) n. U; ~) e; D& `3 {5 _ - }
复制代码发送完HTTP标准头之后,就需要发送内容头(Content-Type),这处的Content-Type为image/jpeg,同样,HTTP标准协议里面image支持的类型远不止jpeg一种,发送完内容头之后就是正文和boundary结尾,这样帧完整的HTTP头发送到指定的TCP GET地址,就会在浏览器中显示刚刚发送的图片: - <pre class="prettyprint lang-cpp" style="box-sizing: border-box; font-family: Monaco, Menlo, Consolas, "Courier New", monospace; font-size: 16px; white-space: pre-wrap; line-height: 1.38462; color: rgb(51, 51, 51); word-break: break-all; background-color: rgb(245, 245, 245); border: 0px; border-radius: 4px; vertical-align: baseline;"> sprintf(buffer, "Content-Type: image/jpeg\r\n" \* f8 r& V8 f3 B
- "Content-Length: %d\r\n" \' Y5 U9 G# Y: A7 x& V; m8 X8 D/ `
- "X-Timestamp: %d.%06d\r\n" \0 R; M' d" C0 w5 E3 V
- "\r\n", frame_size, (int)timestamp.tv_sec, (int)timestamp.tv_usec);
. w9 [* r: Q& S, v# ` - printf("sending intemdiate header\n");
6 n! w" Q4 e+ N& n- M - if(write(fd, buffer, strlen(buffer)) < 0)
- q! o! Y4 @4 D- H7 I( R - break;
. N% G4 u5 H, d4 U9 J - printf("sending frame\n");: m- }/ h4 z4 e+ W
- if(write(fd, frame, frame_size) < 0)
/ h4 v9 N+ Z( n& k; E8 l - break;
/ J5 c: Q# ]5 E+ u - printf("sending boundary\n");
- q' _. `- G k1 } - sprintf(buffer, "\r\n--" BOUNDARY "\r\n");1 e+ f3 f) {4 T( b' O1 I, R
- if(write(fd, buffer, strlen(buffer)) < 0)
2 O; G6 ?2 \, e0 h, P - break;</pre><p style="box-sizing: border-box; border: 0px; font-size: 16px; vertical-align: baseline; line-height: 26px; color: rgb(51, 51, 51); font-family: " helvetica="" neue",="" helvetica,="" tahoma,="" arial,="" "microsoft="" yahei",="" "hiragino="" sans="" gb",="" "wenquanyi="" micro="" hei",="" sans-serif;"=""></p>
复制代码另外需要说明的是,TCP服务器线程在发送MJPEG流的时候是死循环发送的,因此TCP客户端在发送完GET指令之后,就会收到TCP服务器循环发送的图像缓存,TCP客户端会陷入忙等待状态无法再对外发送任何GET或者POST指令,从客户端使用者角度来看的效果就是网页一直在等待。  $ V0 w! g$ c& z% T' B
 4 X& x% w. ^8 Q; N& e
二、UDP上位机UDP发送操作,同样需要先建立UDP Socket:
' O. t* P! q. J/ C1 S* n; U- int UDP_Send_Found(socklen_t* socket_found , struct sockaddr_in *addr , char* ip , int port)
* d' }$ f8 ~ X& s" \8 C - {# J a5 c* L, i
- *socket_found = socket(AF_INET, SOCK_DGRAM, 0);
4 U& E3 \1 D; I. l. D9 n+ u - if(*socket_found == (~0))
5 }: a: L7 L: X7 j4 f, o - {
3 n; M5 y0 z v: u6 V( N: H! m% ? - printf("Create udp send socket failed!\n");% j8 k+ W8 R0 K4 i: S9 M$ N# x
- return -1;
" z, Z- }1 G' y; U - }, b: H: z1 }$ h' o8 z6 A+ ^) S+ C- P
- addr->sin_family = AF_INET;, A; R+ {. s+ w. g6 c
- addr->sin_addr.s_addr = inet_addr(ip);- B+ i+ q! V- i6 `$ p! b7 Q0 Z2 d% A
- addr->sin_port = htons(port);! z# A2 \. Q4 O( @
- memset(addr->sin_zero, 0, 8);! M3 L" }3 a0 _" n9 T
- return 0;% h. q8 z: `$ n p/ @: k
- }
复制代码 * J# P% C/ a* o
7 s8 u) t1 {9 e6 f! H) q
而UDP文件发送则要比HTTP发送简单得多,只需要将文件切片,每一片为固定长度的UDP帧长度,逐帧发送即可:# h. C2 \# j9 r. o0 X" I
* U$ V* G' `1 y% D! {$ o) Y4 d( X
- while(fend > 0). O) ?0 F$ {$ U& X9 o
- {* d1 H$ K# g1 I
- memset(picture.data , 0 , sizeof(picture.data));
# E: z4 y6 R& Z! E - fread(picture.data , UDP_FRAME_LEN , 1, fp);
4 k! ^ Q" g2 I3 ?4 S* D - if(fend >= UDP_FRAME_LEN)8 l% e* L; n1 q# n, k) ~0 I
- {) {- V2 p2 V( F0 I1 m: }6 R: C
- picture.length = UDP_FRAME_LEN;
3 _: M7 w6 h$ ?9 j# d - picture.fin = 0; L0 n- ?1 h9 f0 q8 |
- }
9 Q! h$ w3 ]; |9 R4 O3 ^0 o0 T - else
; M7 K7 E9 {/ j+ m- _ - {
4 y1 m/ V: P: N, ?/ T1 q - picture.length = fend;' ]+ G4 E7 J% h( T" B
- picture.fin = 1;) E& T; C& q: K/ k9 R8 L
- }2 N: P) G, V/ P0 P r7 H
- //printf("sendbytes = %d \n",sendbytes); f6 T: G- C2 t4 g# ? V
- sendbytes = sendto(socket_send, (char *)&picture, sizeof(struct Package), 0, (struct sockaddr*)&addr,addr_len);
`7 Z( `: k/ n3 K3 w& T+ @ - if(sendbytes == -1)/ K6 N& H* b( u# e
- {5 }4 [! r' I3 E& _2 w+ a. g
- printf("Send Picture Failed!d\n");: T' u5 V! B2 {
- return -1;) G1 O( F5 g& X
- }8 Q# ~3 n1 b4 q+ u# t& s
- else; A9 [) b& Z" J; {: N
- {
6 a+ I3 d$ y7 g i( c1 A - fend -= UDP_FRAME_LEN;$ D- n& q1 g0 x3 j5 Y% n8 b3 ]* W
- }" s7 o3 \/ f0 E; ]) m3 J6 \: e0 K
- }
复制代码 ' ?1 j4 X6 y B) J& c4 z3 v
1 c7 ?$ m/ I; { c! {% W7 t i
 5 F, H, W; L2 |4 E+ g0 ?$ U# \
& t, M0 P* f7 L4 V c/ d( C
iMX8MPlus 核心板: https://www.forlinx.com/product/136.html |