本帖最后由 飞凌-marketing 于 2022-1-27 11:00 编辑 # T2 U; j5 {: ^, x, z; q
4 c# @* D: G g# }1 E% N 5 H. B; |9 R9 D& R7 v6 `
作者|donatello1996 来源 | 电子发烧友 题图|飞凌嵌入式 iMX8MPlus 核心板: https://www.forlinx.com/product/136.html8 B& H; }2 W. u1 r
. a4 z' y/ S4 }1 G4 s% K2 J
本文采用的硬件板卡为飞凌嵌入式OKMX8MP-C开发板,系统版本Linux5.4.70+Qt5.15.0,主要介绍基于HTTP网页服务器和UDP上位机的MJPG码流传输。 MJPG格式作为一种持续传输的视频码流,在远程监控领域中应用较广,而实现这种远程监控的第三方应用最常见的有两种:浏览器HTTP网页、UDP上位机。
3 T, A( s4 A3 @! s% F8 I1 f
! n& q/ k8 K+ @* p* _. Y+ i: l两者各有优势,对比鲜明,其中: 这两种应用各有优缺点,对于嵌入式开发者来说,两者都必须掌握。 $ I7 E X0 I+ L# \8 f$ A" Z
一、HTTP网页服务器6 N% u- b+ `9 _1 e4 z l( Y( I
先说下HTTP网页服务器获取MJPG码流的代码,首先是OKMX8MP-C在开发板端建立TCP服务器: - int TCP_Server_Found(socklen_t* socket_found , char* ip , int port); ~7 t4 H9 u W& o
- {5 K7 Z \' F. u2 P. b
- struct sockaddr_in servaddr;
$ f% w& m: c' G1 o+ X. z6 p - socklen_t addrsize = sizeof(struct sockaddr);: p$ t3 \ Q+ U- e* E& _
- bzero(&servaddr , sizeof(servaddr));
3 {+ F6 S) s: r( c - servaddr.sin_family = AF_INET;, t5 k/ d* W2 U y, ]
- servaddr.sin_addr.s_addr = inet_addr(ip);3 ]! A" X7 w% b$ a
- servaddr.sin_port = htons(port);
; a, L1 }- ~: g; F6 G& [8 j' j - int ret;
7 Q# I' H0 ?7 K4 A7 }. g - IF( (*socket_found = socket(AF_INET , SOCK_STREAM , 0)) == -1)
1 \9 C' p$ W5 Q4 I - {
- G% X1 o& W6 ] G2 U# q - printf("Create socket error: %s (errno :%d)\n",strerror(errno),errno);
5 j7 ^7 }% Z$ q# H - return -1;* p) [5 A( Z( {: c3 I" E
- }
% q) r3 m/ q/ @$ A# N) U* N& [ - int on = 1;* S9 k0 y( X$ [0 ]
- if(setsockopt(*socket_found , SOL_SOCKET, SO_REUSEADDR, &on, sizeof(on)) < 0)
( P$ A6 u7 X$ `+ c- ]' J' q: k - {: c& ^/ E1 {5 \
- printf("setsockopt error\n");
$ ~# P+ G" R' E; ^. c3 t+ j- K1 U - }
& M$ g' Y* m4 r3 h" L' }5 E - ret = bind(*socket_found , (struct sockaddr *)&servaddr , addrsize);
# T: a/ W4 T+ \" `( w2 @ - if(ret == -1)
+ J1 j: R3 ^0 C& O/ | - {* ^' c: B6 h' @" y4 y" L, J
- printf("Tcp bind faiLED!\n");2 Q0 w$ P- V+ w/ t
- return -1;4 `* | s3 o' f+ Z
- }* i1 I) x' w+ c! m0 F8 J, v1 W
- if(listen(*socket_found , 5) == -1)
, Q/ U' v( N8 z, H1 y% f5 f - {4 P7 ~1 N8 K: j
- printf("Listen failed!\n");
5 N0 E1 G3 v, P3 H - return -1;/ {, z* L: [! [) j' k
- }
; E! N* ]7 k: b( i5 Q; E - return 0;3 n6 r( J9 ?3 W
- }
复制代码其中setsockopt()函数是可选的,一般只用于规避socket()函数的建立错误。 建立了TCP服务器后,返回的socklen_t型实参在后面的HTTP网页服务器中需要用到。 HTTP网页服务器所属的TCP操作是需要另起轮询线程来让客户端进行accept()握手操作的,accept()之前的listen()倒是只需要执行一次即可,accept()握手操作和recv()接收操作需要创建一个死循环线程: - pthread_create(&tid_tcp_web_recv , NULL , Thread_TCP_Web_Recv , NULL);
" S& }# S, ?: |6 p6 ^ - void * Thread_TCP_Web_Recv(void *arg)
, x& G q; k) @6 U1 g - {
* p* `* ^& E5 u7 R0 X/ H - 。。。
) {' M4 _2 T9 {# g. \. h* j/ x - while(1)+ S- F" s2 d4 Y! _6 n( ~6 i
- {
4 m4 M( T) q5 c5 U' H D' ?( v - fd_socket_conn = accept(socket_web_server , (struct sockaddr *)&sockaddr_in_conn , &addrsize);" [# t0 q: Z- g" m4 z
- printf("fd_socket_conn = accept()\n");' T9 a/ @3 [& Y- L# J
- 。。。; L9 {% l/ S# A1 F$ G
- recv(fd_socket_conn , recvbuf , 1000 , 0);/ k' H4 A' ]* E- b. L) G$ F, a
- }1 U1 c& `8 t6 z6 t" |
- 。。。
; y- M+ b- [! p* o. t* l: X' | - }
复制代码MJPG帧可以使用Grab操作获取,获取到的MJPG帧需要在TCP线程中读,在Grab操作线程中写,这种被多个线程访问的资源需要加锁防止读写冲突,即资源被Grab操作写入时,需要上锁,不允许其它线程访问,操作完成时需要解锁,允许其它线程访问: - pthread_mutex_lock(&pmt);
% \) `( d8 T2 W' M% z" A5 L - pic_tmpbuffer = pic.tmpbuffer;
' O, \% B# R2 p/ V4 p, \1 ? - pic.tmpbytesused = buff.bytesused;
* }1 n4 D! B2 t( V/ j - pic_tmpbytesused = pic.tmpbytesused;7 P) p/ A ?$ B2 G+ T" b' Y
- pthread_cond_broadcast(&pct);0 K! U. J0 v* J6 |" U- i" z
- pthread_mutex_unlock(&pmt);
复制代码线程互斥锁使用之前需要初始化: - pthread_mutex_t pmt;
8 v6 f3 K9 d- }; ^. E4 Y3 c0 u - pthread_cond_t pct;4 C9 O! b" w7 g' c
- int main(int argc, char* argv[])
; z: k' ^* z' E! H2 V) V9 V8 ] - {
0 V4 D# o f, l0 d - ...
6 F A5 _ M' ?% t4 Y# S - TCP_Server_Found(&socket_web_server , (char*)argv[2] , PORT_TCP);9 j0 x4 j4 t4 D/ K) v; z$ E6 D
- pthread_mutex_init(&pmt , NULL);
5 y* U: {# ~2 Y0 ^& A! Z8 F: L - pthread_create(&tid_tcp_web_recv , NULL , Thread_TCP_Web_Recv , NULL);
: k- z) |7 _. p: r# q - pthread_create(&tid_tcp_web_send , NULL , Thread_TCP_Web_Send , NULL);
8 u) M/ ]0 B6 S M U - ..." W- y! Y4 `% F& o- j0 `
- while(1)/ e! v. u! n; ^, S/ Y5 t3 _/ l
- {8 C1 e5 h4 h, ]. u
- V4l2_Grab_Mjpeg(false , MJPEG_FILE_NAME);# T$ E0 x. `& S$ b- g' o! x) k
- ...
* Q( p5 `- w+ L( |8 J - }/ g9 v9 A) D0 k% f$ Q
- ...
- Y T s9 f% z+ k' M1 v - }
复制代码然后是发送的细节,发送图片文件之前,需要先发送HTTP标准头,这个相当于给发送图片或者其它类型的流数据铺路: - <p style="box-sizing: border-box; border: 0px; vertical-align: baseline; line-height: 26px;"><span style="text-indent: 32px;">
+ \$ s w6 ]: y4 m8 { - </span></p><p style="box-sizing: border-box; border: 0px; vertical-align: baseline; line-height: 26px;"></p>
复制代码- #define STD_HEADER "Connection: close\r\n" \
# ]- Q- ]5 U( J. g - "Server: MJPG-Streamer/0.2\r\n" \/ S2 R6 X+ i; g* Y0 M) I w( k
- "Cache-Control: no-store, no-cache, must-revalidate, pre-check=0, post-check=0, max-age=0\r\n" \: m( M$ r3 R9 m% x# D* `
- "Pragma: no-cache\r\n" \
' f1 t k! c, h0 i - "Expires: Mon, 3 Jan 2000 12:34:56 GMT\r\n"
; K9 X/ [! Q& }8 S4 M6 _$ ^ - #define BOUNDARY "boundarydonotcross"2 ~( u$ D) ^3 s: b: B6 }6 I! ^
- printf("preparing header\n");
# v/ I: i/ p/ s9 |8 B3 { - sprintf(buffer, "HTTP/1.0 200 OK\r\n" \) W- w6 v- A# W/ I, }
- "Access-Control-Allow-Origin: *\r\n" \
@& I$ |0 t% ?7 A1 F - STD_HEADER \8 D' ~7 I8 M7 _
- "Content-Type: multipart/x-mixed-replace;boundary=" BOUNDARY "\r\n" \
( W+ j$ o1 n* @- g% y2 {0 Z/ l, S - "\r\n" \ g) ^" y- E) G& O- F: G1 }
- "--" BOUNDARY "\r\n");
' z% M% c( V& E6 ` - if(write(fd, buffer, strlen(buffer)) < 0)4 X7 K, }' F* I1 T6 m" U& E
- {( H$ k" F0 z7 K) [" @
- free(frame);
n; W2 i% ~) A8 _0 } - return;9 L1 M- A& v9 O' r- G; H& `
- }
复制代码发送完HTTP标准头之后,就需要发送内容头(Content-Type),这处的Content-Type为image/jpeg,同样,HTTP标准协议里面image支持的类型远不止jpeg一种,发送完内容头之后就是正文和boundary结尾,这样帧完整的HTTP头发送到指定的TCP GET地址,就会在浏览器中显示刚刚发送的图片: - <pre class="prettyprint lang-cpp" style="box-sizing: border-box; font-family: Monaco, Menlo, Consolas, "Courier New", monospace; font-size: 16px; white-space: pre-wrap; line-height: 1.38462; color: rgb(51, 51, 51); word-break: break-all; background-color: rgb(245, 245, 245); border: 0px; border-radius: 4px; vertical-align: baseline;"> sprintf(buffer, "Content-Type: image/jpeg\r\n" \) C7 T; W! R) z; C
- "Content-Length: %d\r\n" \: y, e4 @, o( s4 W4 t# i1 l1 T6 D
- "X-Timestamp: %d.%06d\r\n" \
; l5 m2 P$ D/ c' D9 H/ L2 L5 R - "\r\n", frame_size, (int)timestamp.tv_sec, (int)timestamp.tv_usec);
# c' H9 w. ^- q' j& l3 Y, p2 E8 Z - printf("sending intemdiate header\n");) S% a( N' ~! [5 W+ U# Z
- if(write(fd, buffer, strlen(buffer)) < 0)
; E5 {- f: Z) G$ V - break;" J! H7 ]& p% `0 |/ ~, b
- printf("sending frame\n");
9 L3 ^0 O1 j8 x, ~" ?8 L. Q - if(write(fd, frame, frame_size) < 0)
5 d3 `' x& |9 F7 z, ]! V - break;
# G8 X! q1 _& V* B2 F$ d2 t* z - printf("sending boundary\n");1 G6 R) x; O B$ t) i3 n' m7 v
- sprintf(buffer, "\r\n--" BOUNDARY "\r\n");) o! A- Z" a+ O! d2 f) N
- if(write(fd, buffer, strlen(buffer)) < 0)+ o- i% M6 _. I& g+ x) C
- break;</pre><p style="box-sizing: border-box; border: 0px; font-size: 16px; vertical-align: baseline; line-height: 26px; color: rgb(51, 51, 51); font-family: " helvetica="" neue",="" helvetica,="" tahoma,="" arial,="" "microsoft="" yahei",="" "hiragino="" sans="" gb",="" "wenquanyi="" micro="" hei",="" sans-serif;"=""></p>
复制代码另外需要说明的是,TCP服务器线程在发送MJPEG流的时候是死循环发送的,因此TCP客户端在发送完GET指令之后,就会收到TCP服务器循环发送的图像缓存,TCP客户端会陷入忙等待状态无法再对外发送任何GET或者POST指令,从客户端使用者角度来看的效果就是网页一直在等待。 
: M- E; e" p8 W! N4 T8 B4 r" i+ h
9 K, v) U) {% e* @( }+ G+ x二、UDP上位机UDP发送操作,同样需要先建立UDP Socket:2 X. h( ?/ ~; s8 m; |( I- x2 Q
- int UDP_Send_Found(socklen_t* socket_found , struct sockaddr_in *addr , char* ip , int port)
" Q* `9 U c5 p8 C - {0 \5 U7 e( Y9 A) w2 S, C
- *socket_found = socket(AF_INET, SOCK_DGRAM, 0);9 R2 T+ F; ]! h& c# S2 R6 q( f7 `
- if(*socket_found == (~0))2 {) r2 A- p P n: _# g# [
- {# v1 A2 r0 `: P$ B' L! Q
- printf("Create udp send socket failed!\n");
/ a: B( s! @4 L - return -1;4 a# W$ Z9 ~5 X# B
- }9 r' ?9 a5 ^) d' m9 L% a
- addr->sin_family = AF_INET;
& `* I$ |; K2 B; H& d - addr->sin_addr.s_addr = inet_addr(ip);% f3 Z7 ]' `' \4 k0 f& }/ J+ g2 [
- addr->sin_port = htons(port);
; j) d c. n$ } - memset(addr->sin_zero, 0, 8);
5 x* A% L& g( N9 t - return 0;
* U& A# o8 F1 R; ]! \) { - }
复制代码 ; q, _ ]7 S1 t% n$ ]9 c% l2 |
. T1 R1 Y# p- ~8 ^- I
而UDP文件发送则要比HTTP发送简单得多,只需要将文件切片,每一片为固定长度的UDP帧长度,逐帧发送即可:% n' s' P: e! \
- K V) O$ T4 s8 N6 D4 L: [2 X
2 w! j# ^/ r5 S! L! F; u- x- while(fend > 0)
! o' c; y' t8 a - {
( D: h! }2 L4 R6 B+ Q - memset(picture.data , 0 , sizeof(picture.data));
9 O, h$ A% f" Q( c; z - fread(picture.data , UDP_FRAME_LEN , 1, fp);0 r) z8 q; {7 g, e8 a. O8 A
- if(fend >= UDP_FRAME_LEN)# B2 A |( i s* W5 a9 B" a
- {8 D+ i3 M" N( Q# d E
- picture.length = UDP_FRAME_LEN;
1 {5 \$ e3 D' _' c3 i; B; q - picture.fin = 0;; t9 X2 k* g2 A* m
- }
6 k$ K5 Z( c, ]7 [ - else
" f, Q. P @) } k1 k - {8 j @, _ G( m* |2 M$ V- n* ~- q
- picture.length = fend;/ v6 |, c- I! |4 z, P, G
- picture.fin = 1;
, u6 N: G1 P( y* Z5 O$ u: u - }2 I# t, z6 @8 V, W: W$ F3 s! p$ G. ~2 e" W
- //printf("sendbytes = %d \n",sendbytes);
* e) B4 s# i7 s0 C/ o$ | - sendbytes = sendto(socket_send, (char *)&picture, sizeof(struct Package), 0, (struct sockaddr*)&addr,addr_len);
" h- }7 x4 j7 s8 K0 x$ m - if(sendbytes == -1)
+ k. I+ l) r2 Z - {
1 b b/ H9 v9 l! P. o - printf("Send Picture Failed!d\n");
9 p& V, V, ~! `6 G - return -1;
" r( [2 o2 }# \, d - }. Q) n3 b! m5 ]: A
- else
$ ?- l9 ~/ R% f - {
7 g4 F p5 b. q& j- F - fend -= UDP_FRAME_LEN;; W6 E0 T- J) f1 ?( A- |
- }
$ [1 j l' x9 Z) R% x5 E) l# ] - }
复制代码 / j: ?+ M. w4 B6 r
! W9 N( `4 G, g" N3 P+ `5 r' ~
! a7 Y' [) K' p* U2 |# T# J/ ?/ Q1 z, l! ?' |# ]
iMX8MPlus 核心板: https://www.forlinx.com/product/136.html |