本帖最后由 飞凌-marketing 于 2022-1-27 11:00 编辑
% D) c5 ^ _9 m1 V1 @+ ^6 }3 {
- N$ C& `6 P: q! o, Q6 C7 a
% h8 U6 Y# \9 B( E( s5 J* m作者|donatello1996 来源 | 电子发烧友 题图|飞凌嵌入式 iMX8MPlus 核心板: https://www.forlinx.com/product/136.html
$ T1 O |' c$ m) m; @8 |% g9 c, ` r) M: Z4 W% Y7 H
本文采用的硬件板卡为飞凌嵌入式OKMX8MP-C开发板,系统版本Linux5.4.70+Qt5.15.0,主要介绍基于HTTP网页服务器和UDP上位机的MJPG码流传输。 MJPG格式作为一种持续传输的视频码流,在远程监控领域中应用较广,而实现这种远程监控的第三方应用最常见的有两种:浏览器HTTP网页、UDP上位机。 5 ^: Q* v- D3 I) n: c

3 F/ I1 X1 C" X两者各有优势,对比鲜明,其中: 这两种应用各有优缺点,对于嵌入式开发者来说,两者都必须掌握。
+ K" V* r! s% n4 X" o2 E4 T一、HTTP网页服务器4 W' M$ V$ T1 N/ Z" X
先说下HTTP网页服务器获取MJPG码流的代码,首先是OKMX8MP-C在开发板端建立TCP服务器: - int TCP_Server_Found(socklen_t* socket_found , char* ip , int port)! Y. A- j( Y! |" ]$ v, u2 J& T
- {. O, R- M" Y- ] h) ~
- struct sockaddr_in servaddr;" _5 t0 U X, y, u, |- a- o. {# b
- socklen_t addrsize = sizeof(struct sockaddr);; d$ \' a' M0 [
- bzero(&servaddr , sizeof(servaddr));3 i. F" e& f' M3 e
- servaddr.sin_family = AF_INET;6 t0 x7 O1 y3 y+ x# M# d
- servaddr.sin_addr.s_addr = inet_addr(ip);( m- F y3 K* j9 s, m
- servaddr.sin_port = htons(port);
. p/ H# K- d3 U2 ]7 B - int ret;
3 F5 V' `/ ^, G0 u - IF( (*socket_found = socket(AF_INET , SOCK_STREAM , 0)) == -1)3 x) I4 E# ?' s) t
- {9 C" o$ j+ c& r; p5 D; R
- printf("Create socket error: %s (errno :%d)\n",strerror(errno),errno);6 L' u7 @0 l3 ^2 A3 S! `
- return -1;
% u. |* s6 k6 f3 W* K - }
1 u: v0 ^, l0 G; } - int on = 1;2 b: X. q( X @# D' e2 y' P* l, X6 |2 M) X
- if(setsockopt(*socket_found , SOL_SOCKET, SO_REUSEADDR, &on, sizeof(on)) < 0)* I5 o) H0 s$ H1 z
- {6 e3 s- r1 M: \. L/ d7 |& T# L3 P
- printf("setsockopt error\n");$ H0 U3 {$ O0 J' N" ]! K! W
- }/ M) P2 R+ P) m. d: }& E0 Y5 }
- ret = bind(*socket_found , (struct sockaddr *)&servaddr , addrsize);% w% t8 G2 N5 o* f
- if(ret == -1)8 O. j o' r1 Q
- {4 D5 q5 [7 E8 x" G5 y# W- o' \3 @
- printf("Tcp bind faiLED!\n");5 w6 d: M' w" w' w" d, p/ b
- return -1;
) x) F( @' {2 Q- |. b( y! B" c8 H - }
5 g6 w" m) {5 H c - if(listen(*socket_found , 5) == -1)
* K( Q& y: ?/ ?5 s) i - {
3 p( L7 e- I1 i5 o) L; L - printf("Listen failed!\n");
8 P8 R2 G; h/ |5 k2 q - return -1;* a5 L0 U7 h. Y: V# E( M& e* P
- }
' D, Q5 v; _* m | o! T( [ - return 0;! e- r* ?& m) l; D9 ]
- }
复制代码其中setsockopt()函数是可选的,一般只用于规避socket()函数的建立错误。 建立了TCP服务器后,返回的socklen_t型实参在后面的HTTP网页服务器中需要用到。 HTTP网页服务器所属的TCP操作是需要另起轮询线程来让客户端进行accept()握手操作的,accept()之前的listen()倒是只需要执行一次即可,accept()握手操作和recv()接收操作需要创建一个死循环线程: - pthread_create(&tid_tcp_web_recv , NULL , Thread_TCP_Web_Recv , NULL);& r% a2 ?3 o. R. w
- void * Thread_TCP_Web_Recv(void *arg)3 _0 Y! N% ]! {. ?
- {5 q: Q0 L: y: i: i# P3 W5 Y7 N9 g
- 。。。2 U& p4 i, i; G8 Y
- while(1)
' o5 N! J8 L1 {" r( W; m1 { - {
& k. J) G$ |+ N/ s) a, x i - fd_socket_conn = accept(socket_web_server , (struct sockaddr *)&sockaddr_in_conn , &addrsize);" @ t: ?) C) P, _9 R0 F8 k) t' w
- printf("fd_socket_conn = accept()\n");
4 r5 b9 ]+ _, G: p+ w- N - 。。。6 `4 D0 @4 j: D1 R/ c3 G6 v" [' f
- recv(fd_socket_conn , recvbuf , 1000 , 0);' P+ l' n0 v" ~, Z
- }
* k& d1 Z# F$ N8 r. m - 。。。; |$ Z" X7 }3 L; d+ ]2 X6 F
- }
复制代码MJPG帧可以使用Grab操作获取,获取到的MJPG帧需要在TCP线程中读,在Grab操作线程中写,这种被多个线程访问的资源需要加锁防止读写冲突,即资源被Grab操作写入时,需要上锁,不允许其它线程访问,操作完成时需要解锁,允许其它线程访问: - pthread_mutex_lock(&pmt);! F6 ~8 T, s6 K2 f. t- c
- pic_tmpbuffer = pic.tmpbuffer;
8 s5 j, `+ @, W) Y$ o/ ?; f; F - pic.tmpbytesused = buff.bytesused;; A& \6 l, m. u
- pic_tmpbytesused = pic.tmpbytesused;
8 N, l; P" n, n4 d - pthread_cond_broadcast(&pct);
7 d- a' Z( P' k - pthread_mutex_unlock(&pmt);
复制代码线程互斥锁使用之前需要初始化: - pthread_mutex_t pmt;& g" C. r: K% J! P& z0 y) q
- pthread_cond_t pct;; h2 C! ~' c! r. V+ ~7 \5 R
- int main(int argc, char* argv[])) D; J# {7 a+ M( Q) G9 i
- {* g1 N6 A( m4 o0 K2 J8 }
- ...
% N' T' J7 w9 t$ e U* A( d - TCP_Server_Found(&socket_web_server , (char*)argv[2] , PORT_TCP); W2 r3 U: @8 _' E* z! o( T
- pthread_mutex_init(&pmt , NULL);
; C2 `# k: h1 }% |8 ?6 l9 ^ - pthread_create(&tid_tcp_web_recv , NULL , Thread_TCP_Web_Recv , NULL);" {( }& ^, b; E4 a% @, V
- pthread_create(&tid_tcp_web_send , NULL , Thread_TCP_Web_Send , NULL);
5 o4 Y$ a' r. ? - ...; H# ~( G# n; F
- while(1)7 f3 y% p$ _) \, ^
- {
+ m: U' J( \( B2 w' L - V4l2_Grab_Mjpeg(false , MJPEG_FILE_NAME);6 [1 ~4 J" Z3 S# q! _6 P9 C
- ...
8 ?/ H- W. ?* F2 n7 [) I& z6 P - }( A L' | R- q! a% x5 q
- ...) _9 ]: a6 ?: v$ q! a. M! V3 L
- }
复制代码然后是发送的细节,发送图片文件之前,需要先发送HTTP标准头,这个相当于给发送图片或者其它类型的流数据铺路: - <p style="box-sizing: border-box; border: 0px; vertical-align: baseline; line-height: 26px;"><span style="text-indent: 32px;">. d8 r2 @- y! k: y4 I
- </span></p><p style="box-sizing: border-box; border: 0px; vertical-align: baseline; line-height: 26px;"></p>
复制代码- #define STD_HEADER "Connection: close\r\n" \* S9 Y/ g) D7 x2 Q. H& N' z" [% H8 }" R
- "Server: MJPG-Streamer/0.2\r\n" \
* x9 U0 p/ o6 u8 I/ M7 m @! R - "Cache-Control: no-store, no-cache, must-revalidate, pre-check=0, post-check=0, max-age=0\r\n" \/ B2 X/ J, {6 ?+ I) U: K a# |
- "Pragma: no-cache\r\n" \
; c9 d& i5 v: L7 Q% B2 | - "Expires: Mon, 3 Jan 2000 12:34:56 GMT\r\n" x+ _ w' k+ j4 ^3 Z4 |( Y
- #define BOUNDARY "boundarydonotcross"
+ G$ c6 `) z7 D } - printf("preparing header\n");6 p ^2 L' k5 G h4 G8 h1 v% c
- sprintf(buffer, "HTTP/1.0 200 OK\r\n" \
8 s* T( p1 Y4 V* P4 X3 `- m7 k0 R - "Access-Control-Allow-Origin: *\r\n" \
, m2 v) y s& q: z/ H - STD_HEADER \$ ^' S0 H" h7 b ?# g
- "Content-Type: multipart/x-mixed-replace;boundary=" BOUNDARY "\r\n" \# m+ A+ P% u0 }- E
- "\r\n" \
b4 o* Q* p0 b8 g) b+ c - "--" BOUNDARY "\r\n");& |6 K# N/ `$ k
- if(write(fd, buffer, strlen(buffer)) < 0)0 D: \6 H% X" U8 t
- {$ L* m: X2 o: @4 L
- free(frame);. Z2 M8 _6 G2 }+ n- g. q- b
- return;
( E; {' N& u1 ?& r( O" ~+ | - }
复制代码发送完HTTP标准头之后,就需要发送内容头(Content-Type),这处的Content-Type为image/jpeg,同样,HTTP标准协议里面image支持的类型远不止jpeg一种,发送完内容头之后就是正文和boundary结尾,这样帧完整的HTTP头发送到指定的TCP GET地址,就会在浏览器中显示刚刚发送的图片: - <pre class="prettyprint lang-cpp" style="box-sizing: border-box; font-family: Monaco, Menlo, Consolas, "Courier New", monospace; font-size: 16px; white-space: pre-wrap; line-height: 1.38462; color: rgb(51, 51, 51); word-break: break-all; background-color: rgb(245, 245, 245); border: 0px; border-radius: 4px; vertical-align: baseline;"> sprintf(buffer, "Content-Type: image/jpeg\r\n" \
+ t9 t2 n* y( U) n | - "Content-Length: %d\r\n" \$ c: @9 ]5 Q9 W5 W! \" `+ y; b
- "X-Timestamp: %d.%06d\r\n" \
: W6 H" N, r+ z2 ? - "\r\n", frame_size, (int)timestamp.tv_sec, (int)timestamp.tv_usec);
& E& k2 J6 }/ s2 U - printf("sending intemdiate header\n");( m. B% N7 w. u% d3 X( s* r
- if(write(fd, buffer, strlen(buffer)) < 0). E# p' ^2 Q1 _; ]9 U3 h3 f5 H" I
- break;5 M f$ y, p/ K" l4 L. k) S& x2 j. B
- printf("sending frame\n");3 J4 y1 `! X7 P1 ~7 j+ _
- if(write(fd, frame, frame_size) < 0): B( b7 W' z0 g( d
- break;$ ^' w" @/ [. }) z7 V( X2 a1 b
- printf("sending boundary\n");
& ?, P) P$ z. {. S - sprintf(buffer, "\r\n--" BOUNDARY "\r\n");
% x$ h, g' M0 O0 z - if(write(fd, buffer, strlen(buffer)) < 0)
! y8 ?! ] W, N - break;</pre><p style="box-sizing: border-box; border: 0px; font-size: 16px; vertical-align: baseline; line-height: 26px; color: rgb(51, 51, 51); font-family: " helvetica="" neue",="" helvetica,="" tahoma,="" arial,="" "microsoft="" yahei",="" "hiragino="" sans="" gb",="" "wenquanyi="" micro="" hei",="" sans-serif;"=""></p>
复制代码另外需要说明的是,TCP服务器线程在发送MJPEG流的时候是死循环发送的,因此TCP客户端在发送完GET指令之后,就会收到TCP服务器循环发送的图像缓存,TCP客户端会陷入忙等待状态无法再对外发送任何GET或者POST指令,从客户端使用者角度来看的效果就是网页一直在等待。  4 I+ d6 ?8 l6 b' v) A- k' G

: k# S+ U/ n0 x3 ]; M二、UDP上位机UDP发送操作,同样需要先建立UDP Socket:
6 W5 m' x7 G* c, U- int UDP_Send_Found(socklen_t* socket_found , struct sockaddr_in *addr , char* ip , int port). d1 }5 s7 R; D- v6 b* h' c
- {% j% }, M5 |& Q: k
- *socket_found = socket(AF_INET, SOCK_DGRAM, 0);
0 \' r8 j% \& Z# I7 X - if(*socket_found == (~0))3 z3 B" q( `, Y, Y
- {2 p5 n w5 H. s R# t
- printf("Create udp send socket failed!\n");/ J: S" X- {. \0 @" E/ Y
- return -1;% H8 u5 y! S- t1 T& M6 ^3 k
- }
& Y, ^! Z _8 u5 ?) j0 _5 D - addr->sin_family = AF_INET;7 C; ]& C+ Y3 u$ _0 Y
- addr->sin_addr.s_addr = inet_addr(ip);
/ V$ p* @/ h/ h% o - addr->sin_port = htons(port);
) O4 S: b# r- E' G - memset(addr->sin_zero, 0, 8);6 D' }: n" y3 A. }, p
- return 0;
4 E+ ]# F0 x6 t9 W0 M1 D0 V - }
复制代码
9 X. l E- G r
3 @- Y+ `0 v: I8 r而UDP文件发送则要比HTTP发送简单得多,只需要将文件切片,每一片为固定长度的UDP帧长度,逐帧发送即可:
4 Q k; u7 r$ j. Y: D
8 R* K4 z3 C* B$ v# o! S
% d# w* e6 k' R. Q. }: P, @- while(fend > 0). _$ d" d, K9 L; M; G) M
- {0 d& W9 r- N6 H5 k- B* ^9 `0 S" |% f3 G
- memset(picture.data , 0 , sizeof(picture.data));7 ?. [; V( e0 s& S9 Y* s' W
- fread(picture.data , UDP_FRAME_LEN , 1, fp);
7 |( p$ J7 @1 o - if(fend >= UDP_FRAME_LEN)4 o( S1 h$ y* b: p8 O8 G
- {1 d3 ]7 J/ A. D' i
- picture.length = UDP_FRAME_LEN;& u& A& Z" C" |5 O% N; n4 s
- picture.fin = 0;
C7 t2 T9 i3 |( { - }4 C3 T$ w. R/ m, k% X: H- ~; y
- else1 h' X( c2 a5 N7 O+ C; b3 H
- {. y( t# G! v7 Q
- picture.length = fend;
8 c' V4 ]: A: Z1 { - picture.fin = 1;
( _! r5 k& Y( ]/ @0 \0 U$ {0 l. g - }
7 K5 ~- E0 X& @* ^+ Q/ v$ v: u' l - //printf("sendbytes = %d \n",sendbytes);% l! d0 ]& t- \ v: C8 R |$ |
- sendbytes = sendto(socket_send, (char *)&picture, sizeof(struct Package), 0, (struct sockaddr*)&addr,addr_len);
' ^) Y1 J$ d' z& e - if(sendbytes == -1), e9 f; [9 V" q
- {2 d% ?/ p& L/ `4 d$ V0 D
- printf("Send Picture Failed!d\n");
( G4 c# i, W; j9 ?, b4 x' Z* m% N2 { - return -1;
: t& o4 F5 Y: L' B% W3 _ - }
: L9 M6 t' ~5 m; M# U. `- t7 Z1 R: J - else
6 P6 |' t2 P+ l - {. d! e2 V, w7 P. H
- fend -= UDP_FRAME_LEN;8 L3 L5 ?( j) |
- }
- a x1 }& ~2 F: ^7 |; e( H/ p - }
复制代码
* d, v4 b) q/ M2 N4 _/ [1 _( m6 q: b6 g
 # A$ n# ~! T- S; M
6 s: b0 N* L( x7 f0 n
iMX8MPlus 核心板: https://www.forlinx.com/product/136.html |