本帖最后由 飞凌-marketing 于 2022-1-27 11:00 编辑 3 m3 f6 @2 U: `6 V0 T5 i
4 |: U, ]) m# V B+ w* {, b7 M+ R% {

( f% x$ ]- Y# S$ n作者|donatello1996 来源 | 电子发烧友 题图|飞凌嵌入式 iMX8MPlus 核心板: https://www.forlinx.com/product/136.html; [& N6 S S* l0 E& d& _0 f/ N
) b; X* F% ^0 h6 [% o h5 a
本文采用的硬件板卡为飞凌嵌入式OKMX8MP-C开发板,系统版本Linux5.4.70+Qt5.15.0,主要介绍基于HTTP网页服务器和UDP上位机的MJPG码流传输。 MJPG格式作为一种持续传输的视频码流,在远程监控领域中应用较广,而实现这种远程监控的第三方应用最常见的有两种:浏览器HTTP网页、UDP上位机。 - S/ G1 `6 ]# F( v" N

4 t! R4 v6 n1 Z" m* N# o两者各有优势,对比鲜明,其中: 这两种应用各有优缺点,对于嵌入式开发者来说,两者都必须掌握。
! X; d" l$ e1 I7 N5 d+ X$ j( n一、HTTP网页服务器$ s2 c$ _' `3 q2 R) a; C/ Q- ^
先说下HTTP网页服务器获取MJPG码流的代码,首先是OKMX8MP-C在开发板端建立TCP服务器: - int TCP_Server_Found(socklen_t* socket_found , char* ip , int port)
+ M4 b* U1 M J - {
4 A6 E2 W9 F/ ]6 E* V0 M9 _ - struct sockaddr_in servaddr;; `' Z; \ M3 u
- socklen_t addrsize = sizeof(struct sockaddr);
. o2 L' M/ l5 h4 T1 F6 L - bzero(&servaddr , sizeof(servaddr));
. Y. s0 g" a- }4 f* Y: R' N7 @ - servaddr.sin_family = AF_INET;
8 S1 g9 A6 ]5 v* m5 u1 { - servaddr.sin_addr.s_addr = inet_addr(ip);
; [ w3 Y8 O; A! R0 o2 P - servaddr.sin_port = htons(port);' L3 V4 j7 L* ?0 k5 H! f0 d
- int ret;( |. ~0 d/ x8 v) q/ T( \( N/ m
- IF( (*socket_found = socket(AF_INET , SOCK_STREAM , 0)) == -1)
- Z! u1 u$ B1 w+ I - {; I% H# l* d6 C& z: i
- printf("Create socket error: %s (errno :%d)\n",strerror(errno),errno);$ D% B- [. C: A# l2 } s) g' }
- return -1;
$ S C1 ?* H" [1 S - }) M0 H$ b6 N! k$ C: o1 ]' `
- int on = 1;# {0 Q" w5 B# f# M
- if(setsockopt(*socket_found , SOL_SOCKET, SO_REUSEADDR, &on, sizeof(on)) < 0)4 @3 S( O w7 P5 T2 y( l& v, h! O
- {! F' u& e8 x1 X8 n' |% w4 _% [: R
- printf("setsockopt error\n");) D6 G$ J3 H) P
- }' o1 R9 Y% `; g: q
- ret = bind(*socket_found , (struct sockaddr *)&servaddr , addrsize);4 Q7 J& d" S+ G% D' x7 s4 U$ G
- if(ret == -1)
n1 I2 Y d8 f' b& d - { T: R# ?9 }) ^+ ?, h. H- J* j' Y
- printf("Tcp bind faiLED!\n"); c( r( q8 H5 o# W7 h: ~1 ?
- return -1;9 ]5 V6 ~; O9 B) V5 j0 }' \1 ~
- }6 @8 \* p Q. U$ x- C, d
- if(listen(*socket_found , 5) == -1)6 ~; Y+ T* L# W; Z, Q. g
- {4 j6 ~; v. @6 i
- printf("Listen failed!\n");
1 M9 K6 S) `! C% J$ j/ D - return -1;+ ^8 u* D; r8 `0 `
- }
( j' ^: R$ w7 H4 Q! E) A# {: Y. m4 e/ O - return 0;
3 P8 p" v4 {3 a7 h4 t4 D - }
复制代码其中setsockopt()函数是可选的,一般只用于规避socket()函数的建立错误。 建立了TCP服务器后,返回的socklen_t型实参在后面的HTTP网页服务器中需要用到。 HTTP网页服务器所属的TCP操作是需要另起轮询线程来让客户端进行accept()握手操作的,accept()之前的listen()倒是只需要执行一次即可,accept()握手操作和recv()接收操作需要创建一个死循环线程: - pthread_create(&tid_tcp_web_recv , NULL , Thread_TCP_Web_Recv , NULL);8 \% h$ L4 o( N% v/ `% d
- void * Thread_TCP_Web_Recv(void *arg)
% w ~2 K& j. M0 e1 |& ]/ c% O - {
; H' `. c) f! m5 F - 。。。
+ l' F$ K2 a( B - while(1)
, u' d* M5 y! i0 Z - {
: b/ ]$ i, p- ?1 z; m* m - fd_socket_conn = accept(socket_web_server , (struct sockaddr *)&sockaddr_in_conn , &addrsize);1 v& R% Q& u" Y
- printf("fd_socket_conn = accept()\n");
* d2 Y4 O3 w; z' A6 s! v - 。。。7 o; S. D( O9 m
- recv(fd_socket_conn , recvbuf , 1000 , 0);$ P6 \% p3 s/ i9 a- U) A
- }7 h& T& }: x A& Y
- 。。。
! g) B6 n; d5 e3 W+ p t - }
复制代码MJPG帧可以使用Grab操作获取,获取到的MJPG帧需要在TCP线程中读,在Grab操作线程中写,这种被多个线程访问的资源需要加锁防止读写冲突,即资源被Grab操作写入时,需要上锁,不允许其它线程访问,操作完成时需要解锁,允许其它线程访问: - pthread_mutex_lock(&pmt);* i3 Q$ p; _5 R* m
- pic_tmpbuffer = pic.tmpbuffer;4 V2 e" T( `8 O8 G4 l
- pic.tmpbytesused = buff.bytesused;
" x+ n q2 q, C* _ - pic_tmpbytesused = pic.tmpbytesused;
0 F& j/ }) X6 F8 O: M* q - pthread_cond_broadcast(&pct);0 V7 I1 a- V* U; m
- pthread_mutex_unlock(&pmt);
复制代码线程互斥锁使用之前需要初始化: - pthread_mutex_t pmt;. J7 j3 p& v0 A! c0 A( @
- pthread_cond_t pct;" I4 M. }4 ^" k- e+ Y
- int main(int argc, char* argv[])$ V/ J8 l% { I: M0 w
- {, N- ?( r& a$ P9 p! U
- ...
$ W3 @$ D1 I( s- v, U - TCP_Server_Found(&socket_web_server , (char*)argv[2] , PORT_TCP);6 p6 b. O; j N0 y
- pthread_mutex_init(&pmt , NULL); t4 n. j8 b/ y# @# |' T
- pthread_create(&tid_tcp_web_recv , NULL , Thread_TCP_Web_Recv , NULL);' i5 q) Y/ d# S
- pthread_create(&tid_tcp_web_send , NULL , Thread_TCP_Web_Send , NULL);. P G' U$ t. Y$ G" Y* V$ b4 H/ ~2 s3 R
- ...4 [/ g# ?0 o1 J1 l' u( p
- while(1)
" c# {4 A% M% d. a# ~. F* F6 D - {
1 L% ~4 P) C% q, a9 [3 \0 U - V4l2_Grab_Mjpeg(false , MJPEG_FILE_NAME);* G5 V) Z* w5 r- i4 J) K
- ...6 M( f0 b) }; N$ N9 O# P
- }
2 ~% J; t1 L6 x! t' {6 S: w - ...4 K6 f/ p2 e) Z' l7 P
- }
复制代码然后是发送的细节,发送图片文件之前,需要先发送HTTP标准头,这个相当于给发送图片或者其它类型的流数据铺路: - <p style="box-sizing: border-box; border: 0px; vertical-align: baseline; line-height: 26px;"><span style="text-indent: 32px;">
4 W& M& `8 D5 F2 R9 i6 M+ Q/ C+ A - </span></p><p style="box-sizing: border-box; border: 0px; vertical-align: baseline; line-height: 26px;"></p>
复制代码- #define STD_HEADER "Connection: close\r\n" \. |+ _7 P: q3 u' _2 |+ I1 A
- "Server: MJPG-Streamer/0.2\r\n" \
. B( L x" ] e9 v" R - "Cache-Control: no-store, no-cache, must-revalidate, pre-check=0, post-check=0, max-age=0\r\n" \
" G. z6 z" x | - "Pragma: no-cache\r\n" \
2 P' K7 U9 X( e2 ` - "Expires: Mon, 3 Jan 2000 12:34:56 GMT\r\n"
3 Z1 |: }( G9 i - #define BOUNDARY "boundarydonotcross"' `$ Y8 t. ]% N. W
- printf("preparing header\n");0 F: I! R, S$ p9 p9 ?8 _
- sprintf(buffer, "HTTP/1.0 200 OK\r\n" \
. A: H" `6 _9 d$ O% i# \ - "Access-Control-Allow-Origin: *\r\n" \4 X% e4 u1 b {( T9 \
- STD_HEADER \
% y# K: x# o& s/ C' b* f( \0 |2 q - "Content-Type: multipart/x-mixed-replace;boundary=" BOUNDARY "\r\n" \; ?( B: \2 D9 v
- "\r\n" \. Z7 J7 Z. l% q: h# V, c; |6 V
- "--" BOUNDARY "\r\n");& `6 h7 h5 D. M$ y' i9 p
- if(write(fd, buffer, strlen(buffer)) < 0)
: L" V1 \ W$ F - {
" W3 n. u5 p3 `/ i - free(frame);
6 R1 b; g8 E7 T" l4 J - return;
7 C8 Y3 P7 n5 N3 b - }
复制代码发送完HTTP标准头之后,就需要发送内容头(Content-Type),这处的Content-Type为image/jpeg,同样,HTTP标准协议里面image支持的类型远不止jpeg一种,发送完内容头之后就是正文和boundary结尾,这样帧完整的HTTP头发送到指定的TCP GET地址,就会在浏览器中显示刚刚发送的图片: - <pre class="prettyprint lang-cpp" style="box-sizing: border-box; font-family: Monaco, Menlo, Consolas, "Courier New", monospace; font-size: 16px; white-space: pre-wrap; line-height: 1.38462; color: rgb(51, 51, 51); word-break: break-all; background-color: rgb(245, 245, 245); border: 0px; border-radius: 4px; vertical-align: baseline;"> sprintf(buffer, "Content-Type: image/jpeg\r\n" \
& N0 X8 e6 f5 g+ l. m6 m - "Content-Length: %d\r\n" \
1 |/ E) Z* U# p# j - "X-Timestamp: %d.%06d\r\n" \/ {2 z+ |- \! C
- "\r\n", frame_size, (int)timestamp.tv_sec, (int)timestamp.tv_usec);
7 ^0 h0 b+ x# ~* G U! O% v" c& E5 c - printf("sending intemdiate header\n");
- x4 t, F3 Z9 x - if(write(fd, buffer, strlen(buffer)) < 0)
, n2 ]: f5 |2 c4 Z) t$ X& l - break;# W. t2 l$ Q- A! W% A3 m
- printf("sending frame\n");
( Z8 E- K3 _. G* t8 ~, k; G - if(write(fd, frame, frame_size) < 0). n. |) z( M) E9 ?
- break;
3 a6 I7 Z2 Q) J; j9 Q - printf("sending boundary\n");. L& \! Q; j( t" ?8 L! ^
- sprintf(buffer, "\r\n--" BOUNDARY "\r\n");
8 g0 r: v! j. j+ W3 Z - if(write(fd, buffer, strlen(buffer)) < 0)
& D4 q8 l, P1 a7 V - break;</pre><p style="box-sizing: border-box; border: 0px; font-size: 16px; vertical-align: baseline; line-height: 26px; color: rgb(51, 51, 51); font-family: " helvetica="" neue",="" helvetica,="" tahoma,="" arial,="" "microsoft="" yahei",="" "hiragino="" sans="" gb",="" "wenquanyi="" micro="" hei",="" sans-serif;"=""></p>
复制代码另外需要说明的是,TCP服务器线程在发送MJPEG流的时候是死循环发送的,因此TCP客户端在发送完GET指令之后,就会收到TCP服务器循环发送的图像缓存,TCP客户端会陷入忙等待状态无法再对外发送任何GET或者POST指令,从客户端使用者角度来看的效果就是网页一直在等待。  $ B+ b$ q7 `1 E6 |8 k

9 o2 r2 z0 j4 b! p1 J. S9 f二、UDP上位机UDP发送操作,同样需要先建立UDP Socket:4 x/ i4 W Z% ]7 r) K
- int UDP_Send_Found(socklen_t* socket_found , struct sockaddr_in *addr , char* ip , int port)6 q% v+ u( y9 c! t
- {
9 a7 U6 Z! {+ ~9 V+ D G6 O& |: @ - *socket_found = socket(AF_INET, SOCK_DGRAM, 0);
) p4 V7 C3 u* {8 L - if(*socket_found == (~0))
5 j: @; F; s) j4 A2 x; V& n - {4 k/ z- `; D6 H
- printf("Create udp send socket failed!\n");) ^$ A! B8 \5 k# J5 n- }2 O3 F
- return -1;% L7 C* G7 k* @' @) \( ] g3 D4 W
- }' _8 z! h1 @0 D8 T! R( B' l* _: n
- addr->sin_family = AF_INET;- p+ G/ g; }7 d) X2 @. p
- addr->sin_addr.s_addr = inet_addr(ip);
( \0 r, t5 I1 i. N. q, O% [: w - addr->sin_port = htons(port);' s& a9 o; Q+ ?
- memset(addr->sin_zero, 0, 8);
0 h y5 l W+ X/ `$ t* Q H - return 0;0 X" e1 T" |7 a& F4 @/ u; s! P
- }
复制代码 & h2 _9 D+ T: l9 d% R0 f
2 q8 P* Q, c) @8 O* m* v% Z而UDP文件发送则要比HTTP发送简单得多,只需要将文件切片,每一片为固定长度的UDP帧长度,逐帧发送即可:
' b+ K) l' R8 j
$ D' ]/ \2 U1 P# m% T. d+ ~. x6 g
7 e! \- }- Q4 z- while(fend > 0)0 h* c; t0 p! _! [# t3 f
- {& e. e: X3 @& X; Y7 }( W
- memset(picture.data , 0 , sizeof(picture.data));
* M) N3 G2 d( E& _8 u# ?# p - fread(picture.data , UDP_FRAME_LEN , 1, fp);
1 R2 P1 w( l; q$ }* H8 W - if(fend >= UDP_FRAME_LEN). H# B7 z5 W: q8 a% g
- {& V& g1 V+ Z# s7 ?
- picture.length = UDP_FRAME_LEN;% v8 A1 E/ q0 ~+ Y2 {- ^. i
- picture.fin = 0;
# Z% { J; J! ` - }
! R9 l$ [* {' [' X4 a+ F2 t - else
. K) R V/ J# J4 c' G: l" k - {. y" U3 T# G" G4 u; T. j' |+ ]% `
- picture.length = fend;, y0 A& v. ^$ ^" c- _; u
- picture.fin = 1;2 h/ W6 g: @/ r- ]
- }2 V7 L7 Q, H: q& D5 z
- //printf("sendbytes = %d \n",sendbytes);
/ h8 r, r7 u2 d8 y; r$ t" s. d - sendbytes = sendto(socket_send, (char *)&picture, sizeof(struct Package), 0, (struct sockaddr*)&addr,addr_len);0 d6 t6 Q7 |0 m! T+ D
- if(sendbytes == -1)! j! a( e0 Q$ W
- {0 Q; G6 p9 Z6 S
- printf("Send Picture Failed!d\n");
* I* c1 O, r3 q N% j - return -1;" C5 u4 g$ D) t/ c }8 ]
- }# j7 r4 A" d3 ]& k" ~% b3 J( ^6 u0 u
- else0 D* t! q) V- @+ P
- {
" g' f) _& B6 H) X1 T - fend -= UDP_FRAME_LEN;
) x. I! m8 E5 c7 @2 m6 F - }6 i2 g% e+ w( i5 V j7 k5 F/ T: O
- }
复制代码 3 e: y% y% N5 _
/ w- Z, j5 v4 K- O" b0 M, h( u
9 p1 |4 z B9 D( {2 X
6 o: @7 p- W6 j" f. f# ?iMX8MPlus 核心板: https://www.forlinx.com/product/136.html |