本帖最后由 飞凌-marketing 于 2022-1-27 11:00 编辑
9 e/ ?6 }, W* h: v' t0 D: l3 I' k c& P1 V

/ @* y/ z7 W; d6 W+ x作者|donatello1996 来源 | 电子发烧友 题图|飞凌嵌入式 iMX8MPlus 核心板: https://www.forlinx.com/product/136.html
7 j& L$ k/ ~) E5 E. N' U9 a+ m8 k& `! l7 s! k/ m q
本文采用的硬件板卡为飞凌嵌入式OKMX8MP-C开发板,系统版本Linux5.4.70+Qt5.15.0,主要介绍基于HTTP网页服务器和UDP上位机的MJPG码流传输。 MJPG格式作为一种持续传输的视频码流,在远程监控领域中应用较广,而实现这种远程监控的第三方应用最常见的有两种:浏览器HTTP网页、UDP上位机。 D- }( u1 n" ?, |. a- C6 \

& s2 F% w2 v5 y+ s, i, r. N2 u1 Z# t两者各有优势,对比鲜明,其中: 这两种应用各有优缺点,对于嵌入式开发者来说,两者都必须掌握。
) T- n% S" O% X一、HTTP网页服务器* d+ g: n+ T; ~5 Y; C
先说下HTTP网页服务器获取MJPG码流的代码,首先是OKMX8MP-C在开发板端建立TCP服务器: - int TCP_Server_Found(socklen_t* socket_found , char* ip , int port)
7 i2 V. Y6 D- n* @. w - {
7 f! y' K% j6 ~/ [, Z' ` - struct sockaddr_in servaddr;2 ^( G0 ^. Y- ^" _) O8 m
- socklen_t addrsize = sizeof(struct sockaddr);
+ u% e% a/ F5 ]( c/ i - bzero(&servaddr , sizeof(servaddr));6 `& a0 J) J0 j9 W4 g
- servaddr.sin_family = AF_INET;# q" P6 m. [4 v) h5 g
- servaddr.sin_addr.s_addr = inet_addr(ip);9 C5 V; X1 o0 p
- servaddr.sin_port = htons(port);4 }9 k# R. h, g5 P1 W- e# p+ h' H
- int ret;5 ?1 [1 |, B* I# i
- IF( (*socket_found = socket(AF_INET , SOCK_STREAM , 0)) == -1)
$ K0 A# Z- D4 M$ a9 ]' Y - {: n# O+ f9 h2 Z: \& g" z5 M% t2 J
- printf("Create socket error: %s (errno :%d)\n",strerror(errno),errno);& ?" R3 h+ N6 l$ y2 |9 b
- return -1;
9 ~1 G4 O" z1 J' G/ h2 t0 k' e - }
8 A7 N* m$ b2 F7 w2 T7 |0 ^7 m - int on = 1;5 l# k% P7 R& z \
- if(setsockopt(*socket_found , SOL_SOCKET, SO_REUSEADDR, &on, sizeof(on)) < 0)
! R/ D# q) o5 I' @ - {+ T6 W5 u! v& _! K' |/ I2 u, }
- printf("setsockopt error\n");: D/ l j) t4 J4 j: M+ T$ ]
- }
4 y/ D1 h7 I1 v - ret = bind(*socket_found , (struct sockaddr *)&servaddr , addrsize);
. Q' g: ~" L3 L0 \6 G. f4 ]& d - if(ret == -1)
7 D$ O- g" u3 y - {; U6 e) K4 F3 C# e. g' n& V) k/ ]
- printf("Tcp bind faiLED!\n");
$ ?7 @# ?3 C8 Q* b1 {! c. d - return -1;& L" N- q5 c8 ~) F
- }. y L: R9 P9 m2 H; F* g
- if(listen(*socket_found , 5) == -1)
5 k/ \& a5 u( Y6 Y6 F/ t3 ]7 [8 f5 X - {
1 \$ ]6 q1 t$ U! C6 U" `5 t1 W) _ - printf("Listen failed!\n");
* [0 c9 c& g- @ - return -1;
4 w$ c L& E. x5 h: c0 W9 ]& f- _; f/ u - }
, R( x4 ]: N/ P) }3 j) }! ~ - return 0;: M$ C8 J# I, Y+ U* X9 O
- }
复制代码其中setsockopt()函数是可选的,一般只用于规避socket()函数的建立错误。 建立了TCP服务器后,返回的socklen_t型实参在后面的HTTP网页服务器中需要用到。 HTTP网页服务器所属的TCP操作是需要另起轮询线程来让客户端进行accept()握手操作的,accept()之前的listen()倒是只需要执行一次即可,accept()握手操作和recv()接收操作需要创建一个死循环线程: - pthread_create(&tid_tcp_web_recv , NULL , Thread_TCP_Web_Recv , NULL);
0 G" | A+ r. s/ q9 }5 X3 i - void * Thread_TCP_Web_Recv(void *arg)8 [. t' {! o2 v" B' U; ?- R
- {
% ^+ O# U3 Y t6 h - 。。。, o9 o8 @4 J' K, T" E/ _
- while(1)8 }3 g9 Y9 d+ j3 n/ [
- {
4 K- p( @' d5 ]% X1 }9 l) K - fd_socket_conn = accept(socket_web_server , (struct sockaddr *)&sockaddr_in_conn , &addrsize);
- c2 o) j( \0 v& J - printf("fd_socket_conn = accept()\n");
- R. _8 @- }" i2 D1 [- w2 U - 。。。) j8 x& h+ _$ M$ c9 ]: k% t
- recv(fd_socket_conn , recvbuf , 1000 , 0);
, L5 z7 Y* T6 w# U/ l* @ - }1 h8 \! {; _/ S3 z& u% y L
- 。。。* u+ T4 x2 d+ C* ^6 ?- H/ o# `
- }
复制代码MJPG帧可以使用Grab操作获取,获取到的MJPG帧需要在TCP线程中读,在Grab操作线程中写,这种被多个线程访问的资源需要加锁防止读写冲突,即资源被Grab操作写入时,需要上锁,不允许其它线程访问,操作完成时需要解锁,允许其它线程访问: - pthread_mutex_lock(&pmt);9 i+ P: y# R) C) A1 q8 V
- pic_tmpbuffer = pic.tmpbuffer;
8 I4 N( h4 ?- H9 X j - pic.tmpbytesused = buff.bytesused;
" g8 ~+ \6 Z8 N8 v& | - pic_tmpbytesused = pic.tmpbytesused;
5 S' t b- e5 y, F1 W" y9 ~ - pthread_cond_broadcast(&pct);0 |+ I/ C( i$ x; X* V: r% u$ M% s
- pthread_mutex_unlock(&pmt);
复制代码线程互斥锁使用之前需要初始化: - pthread_mutex_t pmt;! E* K, }9 \1 \! {9 U* P7 D
- pthread_cond_t pct;
, @' y: J& b! ]) \+ ` ? - int main(int argc, char* argv[])9 V8 q: V5 Y) n/ S8 a
- {
8 M/ h4 Z f- g5 |7 i - ...( t0 n r7 K+ x/ z% q* b
- TCP_Server_Found(&socket_web_server , (char*)argv[2] , PORT_TCP);6 `+ F2 D" g& b/ Y1 Z
- pthread_mutex_init(&pmt , NULL);
( g4 c* }" m4 S# ~7 N- ~ - pthread_create(&tid_tcp_web_recv , NULL , Thread_TCP_Web_Recv , NULL);3 v# `3 s2 E, `. M$ [1 ?
- pthread_create(&tid_tcp_web_send , NULL , Thread_TCP_Web_Send , NULL);
8 A' u$ m; ^8 g& S9 H) w9 u - ...& G# \9 m' [: O5 Y
- while(1)) e E9 W( f4 m, a5 s$ \
- {3 A0 X. o. Q+ `: c
- V4l2_Grab_Mjpeg(false , MJPEG_FILE_NAME);; ]6 T- O8 ?( ?: c
- ...9 G5 _2 }* q' s* F- }" s
- }( R* A5 R+ v& j( O0 w2 U
- ...% X# W# D. r+ R7 Y' u
- }
复制代码然后是发送的细节,发送图片文件之前,需要先发送HTTP标准头,这个相当于给发送图片或者其它类型的流数据铺路: - <p style="box-sizing: border-box; border: 0px; vertical-align: baseline; line-height: 26px;"><span style="text-indent: 32px;">9 E& K ]/ N! r( d. C! A: [1 ?* c
- </span></p><p style="box-sizing: border-box; border: 0px; vertical-align: baseline; line-height: 26px;"></p>
复制代码- #define STD_HEADER "Connection: close\r\n" \
; @; j. u& r) P: w0 i: h - "Server: MJPG-Streamer/0.2\r\n" \. k7 R* [) t) A' L& |3 g
- "Cache-Control: no-store, no-cache, must-revalidate, pre-check=0, post-check=0, max-age=0\r\n" \
" l! q. A \. \ - "Pragma: no-cache\r\n" \
# P) v% s* {* Z/ I( a$ } - "Expires: Mon, 3 Jan 2000 12:34:56 GMT\r\n"9 ?+ Z: ^1 r% m( s2 P5 o
- #define BOUNDARY "boundarydonotcross"# W2 u/ V- a x+ e& P' W; l& g
- printf("preparing header\n");
0 G$ h d( d7 X, g8 U$ q( T9 z. X - sprintf(buffer, "HTTP/1.0 200 OK\r\n" \) A, [9 {/ s: q, K0 Q6 K
- "Access-Control-Allow-Origin: *\r\n" \/ @4 ~2 s6 B! g9 F1 V% ]! L' S$ q
- STD_HEADER \/ | Y3 \. [" Z
- "Content-Type: multipart/x-mixed-replace;boundary=" BOUNDARY "\r\n" \. K, C" x! d2 A+ P9 t, s1 n/ }
- "\r\n" \9 i) x6 A$ _& U' Y# y
- "--" BOUNDARY "\r\n");6 S( g9 N- G. Q% f2 e# s4 s1 i
- if(write(fd, buffer, strlen(buffer)) < 0). O3 c$ [" _% k; a9 \5 H, f3 k' D
- {
/ u7 I) Q* I9 b2 L7 m; e - free(frame);7 @5 N$ z' l$ m" M( {' k9 T
- return;; o3 m% Q2 `( d/ f/ }
- }
复制代码发送完HTTP标准头之后,就需要发送内容头(Content-Type),这处的Content-Type为image/jpeg,同样,HTTP标准协议里面image支持的类型远不止jpeg一种,发送完内容头之后就是正文和boundary结尾,这样帧完整的HTTP头发送到指定的TCP GET地址,就会在浏览器中显示刚刚发送的图片: - <pre class="prettyprint lang-cpp" style="box-sizing: border-box; font-family: Monaco, Menlo, Consolas, "Courier New", monospace; font-size: 16px; white-space: pre-wrap; line-height: 1.38462; color: rgb(51, 51, 51); word-break: break-all; background-color: rgb(245, 245, 245); border: 0px; border-radius: 4px; vertical-align: baseline;"> sprintf(buffer, "Content-Type: image/jpeg\r\n" \
. e2 a* G+ c) X7 R - "Content-Length: %d\r\n" \2 C v$ w" V$ Z- b0 M
- "X-Timestamp: %d.%06d\r\n" \
3 i" j# z& M2 i8 V8 v" b9 J - "\r\n", frame_size, (int)timestamp.tv_sec, (int)timestamp.tv_usec);
; p/ p" T, Z* J! Y, g+ c - printf("sending intemdiate header\n");
. U* T5 d8 t" S% c - if(write(fd, buffer, strlen(buffer)) < 0)* }9 F7 d# ?" x6 \0 A. O1 _7 Y
- break;" n7 V, |: j: e1 E% ^
- printf("sending frame\n");
5 p( H7 j; b" Z" M; B - if(write(fd, frame, frame_size) < 0)
# \0 A6 O- b4 e - break;; E5 T" a# f( [& r8 c" ~" ]3 b3 e) j1 p
- printf("sending boundary\n");$ c$ K! d: O" ]* }! @ ^
- sprintf(buffer, "\r\n--" BOUNDARY "\r\n");
! S% S5 e% x$ C2 Q" t& B0 `3 e5 }- ~ - if(write(fd, buffer, strlen(buffer)) < 0)& P, H4 R9 `! \, I6 C! F
- break;</pre><p style="box-sizing: border-box; border: 0px; font-size: 16px; vertical-align: baseline; line-height: 26px; color: rgb(51, 51, 51); font-family: " helvetica="" neue",="" helvetica,="" tahoma,="" arial,="" "microsoft="" yahei",="" "hiragino="" sans="" gb",="" "wenquanyi="" micro="" hei",="" sans-serif;"=""></p>
复制代码另外需要说明的是,TCP服务器线程在发送MJPEG流的时候是死循环发送的,因此TCP客户端在发送完GET指令之后,就会收到TCP服务器循环发送的图像缓存,TCP客户端会陷入忙等待状态无法再对外发送任何GET或者POST指令,从客户端使用者角度来看的效果就是网页一直在等待。 
9 ^# T% O5 ^ ?) j& g+ {6 R4 V
5 b; e7 a/ x @! [ T二、UDP上位机UDP发送操作,同样需要先建立UDP Socket:0 v- J6 {; Q; `% i! e
- int UDP_Send_Found(socklen_t* socket_found , struct sockaddr_in *addr , char* ip , int port)( m; G" |, p2 G" {
- {
3 s4 H% y5 p& H7 c/ E! y1 ~* j - *socket_found = socket(AF_INET, SOCK_DGRAM, 0);/ Y' o, e n8 {4 e( h
- if(*socket_found == (~0))
0 x' q# q4 n3 N. V - {1 p$ _, ~2 B; G, j1 i6 W( q6 L
- printf("Create udp send socket failed!\n");1 r6 d {" d1 u2 U% A7 S% C
- return -1;, U4 J2 Q5 _& Y" `/ r
- }: I. Q1 W2 V9 A, ~
- addr->sin_family = AF_INET;
0 ?: h) X! J3 p0 I4 k - addr->sin_addr.s_addr = inet_addr(ip);
* J4 {4 D% b! Z0 Z7 ~ - addr->sin_port = htons(port);2 m( r, K# I9 m1 {0 e. T
- memset(addr->sin_zero, 0, 8);
! M9 @3 G% V. i - return 0;) N* q3 l1 i' \: G! D K7 x% m
- }
复制代码
0 E# n2 ^ i' V9 V6 T* \, _& n
" m" ]1 j: c8 E1 ?/ Z, d而UDP文件发送则要比HTTP发送简单得多,只需要将文件切片,每一片为固定长度的UDP帧长度,逐帧发送即可:
" Z1 h2 c) F" y- ~' c F- k3 \9 X' b) r! s
. Q- m) e- i; s; o/ j
- while(fend > 0)
: G2 {6 r( X, K( _ - {3 e1 X" _ C' W6 Z Z4 Y. N
- memset(picture.data , 0 , sizeof(picture.data));
. S$ G/ Z" g6 ^. c `; c2 @6 ~ - fread(picture.data , UDP_FRAME_LEN , 1, fp);
( ]/ I3 {0 ~3 I/ @8 ]( k - if(fend >= UDP_FRAME_LEN)& X; X7 H2 C2 V& V# Y
- {
4 P; `- n; r/ l) W - picture.length = UDP_FRAME_LEN;& D( [3 J8 X0 C" w: C) k- T
- picture.fin = 0;- f" n8 t& Q- |" o. Z
- }% |: O2 u# g, |! J+ Z( I
- else
' h: |, H; p. ]3 E1 g$ v. l - {
) t$ Y5 {6 m6 D& Q- I! J/ p - picture.length = fend;' k/ x; F5 b9 X
- picture.fin = 1;5 D4 Y8 J7 g) F* W2 A$ ]7 T! [* G
- }
2 y0 n, ?& |9 n% O! J% M, M C - //printf("sendbytes = %d \n",sendbytes);
4 h" `5 J+ h l) t - sendbytes = sendto(socket_send, (char *)&picture, sizeof(struct Package), 0, (struct sockaddr*)&addr,addr_len);
1 x4 e& g/ I# Z* |$ M# A- E8 W: Y - if(sendbytes == -1)
: Y9 j: e4 e4 g - {: ^. k' ~% l5 i: y# {
- printf("Send Picture Failed!d\n");& `6 l- R) Z) `+ {
- return -1;, F" U8 _, S( X. A$ l
- }' g0 v7 v+ _$ D; @
- else
1 _( U H2 J3 k- M8 C6 S - {' S/ X2 u- @, i# s+ ^
- fend -= UDP_FRAME_LEN;
! O5 }% P) e& K0 |+ u. ~ - }
/ V1 S" X( h& b3 G! N, G4 k6 k9 u - }
复制代码 & _. o# Z' l# V" g1 }/ N' _
3 I8 q" g6 e' P6 K0 ^
 $ O! k2 ]' L _0 p3 e3 y4 U
F, a# }6 H; S) p" c
iMX8MPlus 核心板: https://www.forlinx.com/product/136.html |