本帖最后由 飞凌-marketing 于 2022-1-27 11:00 编辑 " O J- {" i. j. t+ C6 `
& T1 L. r4 }0 ]" E
 8 U2 s9 e# g- l
作者|donatello1996 来源 | 电子发烧友 题图|飞凌嵌入式 iMX8MPlus 核心板: https://www.forlinx.com/product/136.html' X1 b0 ~6 p/ W/ e, r
3 H! X3 Q2 ]6 P' @- j; ]; B. Q( v
本文采用的硬件板卡为飞凌嵌入式OKMX8MP-C开发板,系统版本Linux5.4.70+Qt5.15.0,主要介绍基于HTTP网页服务器和UDP上位机的MJPG码流传输。 MJPG格式作为一种持续传输的视频码流,在远程监控领域中应用较广,而实现这种远程监控的第三方应用最常见的有两种:浏览器HTTP网页、UDP上位机。 r0 c8 a( ?: j& ~% F
 , y4 G+ q8 ?& T$ F# I# B
两者各有优势,对比鲜明,其中: 这两种应用各有优缺点,对于嵌入式开发者来说,两者都必须掌握。 ' y& Y# y5 { m! k
一、HTTP网页服务器
) e: n" k& h. R7 S) b先说下HTTP网页服务器获取MJPG码流的代码,首先是OKMX8MP-C在开发板端建立TCP服务器: - int TCP_Server_Found(socklen_t* socket_found , char* ip , int port)
1 w/ i+ ]7 y' }$ S* w. ~, e - {" x" v- A, `3 b$ H% X1 V
- struct sockaddr_in servaddr;% l/ s, l' l! x
- socklen_t addrsize = sizeof(struct sockaddr);
% d& l& `0 W% g5 {4 H0 P* Q - bzero(&servaddr , sizeof(servaddr));+ l* \. b7 k- R4 t& R
- servaddr.sin_family = AF_INET;6 c( _2 q2 J/ q Z4 V% U7 Y; J! v
- servaddr.sin_addr.s_addr = inet_addr(ip);
4 G: e/ ]3 J4 P H } - servaddr.sin_port = htons(port);
/ P# F8 j3 B$ T7 z - int ret;
# _ p1 I4 D6 d8 l/ ?7 _' _ - IF( (*socket_found = socket(AF_INET , SOCK_STREAM , 0)) == -1)
& u* f0 m/ {0 b* v4 d, u - {
3 y$ Y& R6 T" V W% u/ U - printf("Create socket error: %s (errno :%d)\n",strerror(errno),errno);
, u2 Z% N+ Z4 X - return -1;
% n; c, \# @% d8 |3 p! F- { - }0 t6 P( X6 ~" \. z7 m
- int on = 1;
* x1 {2 { t# T- J) B+ X - if(setsockopt(*socket_found , SOL_SOCKET, SO_REUSEADDR, &on, sizeof(on)) < 0)
/ F& J* Y' n, U+ i" W' R - {
" c; d" Y6 X1 z' {" M - printf("setsockopt error\n");
0 O# ~( f3 t _8 d3 x9 C - }
C- R5 `1 M! A" i. H0 c' g - ret = bind(*socket_found , (struct sockaddr *)&servaddr , addrsize);% ]% b' G+ X2 L4 U, O
- if(ret == -1)
- x2 O9 m6 {$ N) x - {( P6 V1 L G/ x$ H4 F; c
- printf("Tcp bind faiLED!\n");4 a" I% S9 K6 ^7 ?. R0 Y2 Z, h/ i: O
- return -1;
2 e7 E# Z- e# L" h6 o - }. ^$ B( }: b: j1 L% T3 Q% [6 m5 z
- if(listen(*socket_found , 5) == -1)
5 Q' f8 o5 O. x# T5 E - {
" b% M( k6 z* w/ w9 o: s - printf("Listen failed!\n");
4 s8 a/ b8 m) b7 @ o - return -1;
- R# W1 d" P5 p& l. z8 W% N5 d - }
/ ~$ j, [3 r9 G* o2 E5 _# g& h - return 0;
' w# L& @2 k2 e& y0 l" l9 g9 v, J - }
复制代码其中setsockopt()函数是可选的,一般只用于规避socket()函数的建立错误。 建立了TCP服务器后,返回的socklen_t型实参在后面的HTTP网页服务器中需要用到。 HTTP网页服务器所属的TCP操作是需要另起轮询线程来让客户端进行accept()握手操作的,accept()之前的listen()倒是只需要执行一次即可,accept()握手操作和recv()接收操作需要创建一个死循环线程: - pthread_create(&tid_tcp_web_recv , NULL , Thread_TCP_Web_Recv , NULL);
8 V8 k" m' Q5 T* ]8 T" \ - void * Thread_TCP_Web_Recv(void *arg)2 `' ]9 N/ r7 [. a r
- {7 l! q/ p/ K9 a2 ]- ?
- 。。。
/ O0 q3 D* U/ Z m - while(1)
( z! y4 I8 w Z9 s/ B - {
( B* k6 ~5 K+ q% A3 {3 k( [ - fd_socket_conn = accept(socket_web_server , (struct sockaddr *)&sockaddr_in_conn , &addrsize);
0 ~# V- B" F+ z! w5 O - printf("fd_socket_conn = accept()\n");
4 E* E# |! _# c3 ^, B( G7 f$ [ - 。。。; V9 _/ \* a2 F$ x" c0 _
- recv(fd_socket_conn , recvbuf , 1000 , 0);
' }8 q" r+ q+ Y! K0 [- X - }) [1 e; J5 S$ x
- 。。。
, ^5 c- q4 `; }4 V }$ X% F' ~. @ - }
复制代码MJPG帧可以使用Grab操作获取,获取到的MJPG帧需要在TCP线程中读,在Grab操作线程中写,这种被多个线程访问的资源需要加锁防止读写冲突,即资源被Grab操作写入时,需要上锁,不允许其它线程访问,操作完成时需要解锁,允许其它线程访问: - pthread_mutex_lock(&pmt);. e7 | h0 s! e* C/ f
- pic_tmpbuffer = pic.tmpbuffer;
4 c0 `' g4 \+ N: M7 D - pic.tmpbytesused = buff.bytesused;
$ N i5 c$ ]- s5 Y1 v% |8 N - pic_tmpbytesused = pic.tmpbytesused;
8 }5 S+ g$ l. i E* C - pthread_cond_broadcast(&pct); ?0 y7 d9 T, H& ^$ c; m8 o( L8 n2 p
- pthread_mutex_unlock(&pmt);
复制代码线程互斥锁使用之前需要初始化: - pthread_mutex_t pmt;* o, X$ S1 m# j. ^' m
- pthread_cond_t pct;* A- B4 o+ J3 i
- int main(int argc, char* argv[]) F' d5 c7 }1 R5 l. _- Z
- {) [! j- I6 }7 ^" i2 M
- ...3 ]0 e4 \0 l f0 e8 \* j( E3 M) ^1 p
- TCP_Server_Found(&socket_web_server , (char*)argv[2] , PORT_TCP);2 b/ d/ Y/ H5 l0 W7 z
- pthread_mutex_init(&pmt , NULL);
1 V; l9 ^5 W" ]% ~+ f& s - pthread_create(&tid_tcp_web_recv , NULL , Thread_TCP_Web_Recv , NULL);
5 Z8 w% c1 i- m+ V9 ] - pthread_create(&tid_tcp_web_send , NULL , Thread_TCP_Web_Send , NULL);
3 c' t, E6 K) O4 ~* J, S. J% \6 N - ...
. R7 a, Q( V& r: V - while(1)
& o9 M% `! Z: j( E5 _2 B6 g4 ` - {
- F$ y. G# T& z @! v - V4l2_Grab_Mjpeg(false , MJPEG_FILE_NAME);
1 m8 y- r( K7 h" U& R2 p - ...' z8 L8 U8 P( U1 f4 W. R+ y1 k
- }
; v$ W2 s9 y3 A7 X! H/ H0 f% P3 u# M' ] - ...% v8 V0 V3 V5 s+ [& a
- }
复制代码然后是发送的细节,发送图片文件之前,需要先发送HTTP标准头,这个相当于给发送图片或者其它类型的流数据铺路: - <p style="box-sizing: border-box; border: 0px; vertical-align: baseline; line-height: 26px;"><span style="text-indent: 32px;">
' W) J) R2 [: }! [0 Z. M: Q( b( O - </span></p><p style="box-sizing: border-box; border: 0px; vertical-align: baseline; line-height: 26px;"></p>
复制代码- #define STD_HEADER "Connection: close\r\n" \- t2 U9 o. k. b9 b8 Q& H, C
- "Server: MJPG-Streamer/0.2\r\n" \
6 Q% d! s, `2 D6 O) P$ _$ ?; y6 Z - "Cache-Control: no-store, no-cache, must-revalidate, pre-check=0, post-check=0, max-age=0\r\n" \* }6 b3 r4 D& Z7 a
- "Pragma: no-cache\r\n" \
: i w3 F# k2 H; T; r) N - "Expires: Mon, 3 Jan 2000 12:34:56 GMT\r\n"# U0 c2 W: ~8 S$ G. X9 ]! X
- #define BOUNDARY "boundarydonotcross") [. W" W" @: X F9 L7 ^- D0 m% |, N) @7 a
- printf("preparing header\n");; `! z* `) N( [. J# _: J
- sprintf(buffer, "HTTP/1.0 200 OK\r\n" \6 r- c' y5 R3 b
- "Access-Control-Allow-Origin: *\r\n" \" Y6 P1 _3 p+ f; R" m
- STD_HEADER \
& q6 [; E5 c/ P/ c& J1 `+ C - "Content-Type: multipart/x-mixed-replace;boundary=" BOUNDARY "\r\n" \
- @3 p8 N% p$ |! Y2 H, \ - "\r\n" \
, M! @$ ^% s7 @" m - "--" BOUNDARY "\r\n");
$ V. i. S. H" \8 W - if(write(fd, buffer, strlen(buffer)) < 0)+ ~* B, J1 r! r% g4 P- M
- {
: P% Y5 A. ^( } - free(frame);% s( x. W3 Y1 e8 X
- return;
" z( j- g }7 h* Y q* q( d9 X - }
复制代码发送完HTTP标准头之后,就需要发送内容头(Content-Type),这处的Content-Type为image/jpeg,同样,HTTP标准协议里面image支持的类型远不止jpeg一种,发送完内容头之后就是正文和boundary结尾,这样帧完整的HTTP头发送到指定的TCP GET地址,就会在浏览器中显示刚刚发送的图片: - <pre class="prettyprint lang-cpp" style="box-sizing: border-box; font-family: Monaco, Menlo, Consolas, "Courier New", monospace; font-size: 16px; white-space: pre-wrap; line-height: 1.38462; color: rgb(51, 51, 51); word-break: break-all; background-color: rgb(245, 245, 245); border: 0px; border-radius: 4px; vertical-align: baseline;"> sprintf(buffer, "Content-Type: image/jpeg\r\n" \$ l7 v+ g( n. Z$ J' U
- "Content-Length: %d\r\n" \# g; W/ i# D% T' x) w/ @- r% G
- "X-Timestamp: %d.%06d\r\n" \ x) l8 }7 [4 p4 ~2 x. D
- "\r\n", frame_size, (int)timestamp.tv_sec, (int)timestamp.tv_usec);% Z" P6 p0 i" c6 I1 H' A9 y
- printf("sending intemdiate header\n");
1 H' B3 ]1 s) E: ^9 ~- p# J/ X - if(write(fd, buffer, strlen(buffer)) < 0)
2 B1 ]3 R4 N b" z# N0 | - break;
9 G! Q& K9 L* }9 Q" w - printf("sending frame\n");
$ W( V# [; i5 M; h3 |9 [1 q4 ?7 ] - if(write(fd, frame, frame_size) < 0)2 p$ j: D& Y3 `! I: U0 l7 \1 ~
- break;, F8 z+ E3 R7 ]& \3 ?9 I8 V
- printf("sending boundary\n");
; F8 ]* ]: l ~& J5 _5 Q" B - sprintf(buffer, "\r\n--" BOUNDARY "\r\n");
3 e- m: Z/ ~! d) ~% i - if(write(fd, buffer, strlen(buffer)) < 0)
; N% m# _) a+ f- U - break;</pre><p style="box-sizing: border-box; border: 0px; font-size: 16px; vertical-align: baseline; line-height: 26px; color: rgb(51, 51, 51); font-family: " helvetica="" neue",="" helvetica,="" tahoma,="" arial,="" "microsoft="" yahei",="" "hiragino="" sans="" gb",="" "wenquanyi="" micro="" hei",="" sans-serif;"=""></p>
复制代码另外需要说明的是,TCP服务器线程在发送MJPEG流的时候是死循环发送的,因此TCP客户端在发送完GET指令之后,就会收到TCP服务器循环发送的图像缓存,TCP客户端会陷入忙等待状态无法再对外发送任何GET或者POST指令,从客户端使用者角度来看的效果就是网页一直在等待。  8 }* U. U) i" [
 ' I1 c: J" H$ b' W$ P6 y
二、UDP上位机UDP发送操作,同样需要先建立UDP Socket:
) Q6 Z# z5 K9 B! L6 X- int UDP_Send_Found(socklen_t* socket_found , struct sockaddr_in *addr , char* ip , int port)" v) Y$ y" }9 X
- {) {! g3 h1 L7 @6 Q+ Z
- *socket_found = socket(AF_INET, SOCK_DGRAM, 0);
1 i7 d/ g, D1 f2 p) ?# e+ m( `: k - if(*socket_found == (~0))2 o: x. |$ t; e- ?! @ c
- {. w6 G2 K* L# c, [0 r* p, {4 l
- printf("Create udp send socket failed!\n");
) B7 R7 G' Z6 X. o+ u8 d - return -1;
1 v# c% ?0 P9 D5 |* Z - }6 I, e* P. T F% F9 y
- addr->sin_family = AF_INET;3 [+ ]$ S5 J8 P; C
- addr->sin_addr.s_addr = inet_addr(ip);) j. U2 g0 g) m: L; O0 ^" m
- addr->sin_port = htons(port);9 z8 _0 J+ V) r& g" a8 h" f" s
- memset(addr->sin_zero, 0, 8); H) S- o: a) x5 d0 N
- return 0;- g+ X. M- o. E" g( P
- }
复制代码 - e; B+ t( q# f# G
- ?5 e* R" f3 E4 t+ W8 _6 R* w* F而UDP文件发送则要比HTTP发送简单得多,只需要将文件切片,每一片为固定长度的UDP帧长度,逐帧发送即可:
% }6 N4 q& t9 `# ~& n6 z' e
! s+ E; [) s& [: ~+ u# O: p: ^) M
- while(fend > 0)
4 g' _. I d8 c @* n3 ^ - {
0 N! L. f7 M: r& F# G - memset(picture.data , 0 , sizeof(picture.data));
4 j$ | C1 {. f: m6 ^ - fread(picture.data , UDP_FRAME_LEN , 1, fp);
/ e. f0 s' y5 M, ?- c/ \ - if(fend >= UDP_FRAME_LEN)
, f6 w0 Z8 p5 e& W/ A7 f - {7 g# e) |3 W% `, F
- picture.length = UDP_FRAME_LEN;; }0 E& a* x) f) F# V
- picture.fin = 0;: c8 l6 Y# V0 p) N( V# v* b Q( a
- }1 s( P9 R5 b" |, r# Y0 I8 s# Y# t6 B6 z; \
- else7 ?- \0 w0 z( o5 I1 }
- { {0 H* \$ g0 X9 k" H/ p+ v: l( P
- picture.length = fend;
9 I/ j b' d, k9 [8 `4 F2 E - picture.fin = 1;: i5 K% U. J! s, G
- }
) y! c" l+ I! z$ X6 Y, K - //printf("sendbytes = %d \n",sendbytes);
/ |9 S5 @$ t! n8 I* A9 f4 e' P - sendbytes = sendto(socket_send, (char *)&picture, sizeof(struct Package), 0, (struct sockaddr*)&addr,addr_len);0 B$ ^- m. `2 r1 K
- if(sendbytes == -1)
$ _" [; i; g; }% C - {
: Q9 G* {: p _& p - printf("Send Picture Failed!d\n");
" _# a3 @0 l: P- H - return -1;
+ G2 g" h2 e3 r* e+ J - }. I3 U d5 p4 V' M! _
- else' l, N: {" p" t* ^5 c) k3 I
- {
( I, p& l k. o* L, S4 H - fend -= UDP_FRAME_LEN;
' p7 _4 w4 h0 y7 z- q( w - }5 S! }, A' U l' B; @5 h' C
- }
复制代码
, r: n" j5 N9 p
: z3 W6 {5 u" C 5 @4 z. r. y( n$ v9 s8 h
5 @5 F0 l2 J# ?iMX8MPlus 核心板: https://www.forlinx.com/product/136.html |