本帖最后由 飞凌-marketing 于 2022-1-27 11:00 编辑
2 Q6 K* L) Q* a1 `% n8 Z( Q) z& b! x
 : o: S( P2 |& | ?) t. Y
作者|donatello1996 来源 | 电子发烧友 题图|飞凌嵌入式 iMX8MPlus 核心板: https://www.forlinx.com/product/136.html
s" Z. n( s- v* h a, u$ c* H1 K: h% T4 f+ q. M2 }) X
本文采用的硬件板卡为飞凌嵌入式OKMX8MP-C开发板,系统版本Linux5.4.70+Qt5.15.0,主要介绍基于HTTP网页服务器和UDP上位机的MJPG码流传输。 MJPG格式作为一种持续传输的视频码流,在远程监控领域中应用较广,而实现这种远程监控的第三方应用最常见的有两种:浏览器HTTP网页、UDP上位机。 1 @) J$ r1 |5 C1 N; V2 h

7 ]9 F6 h9 [( \: S4 m" ?1 W- S9 F. d两者各有优势,对比鲜明,其中: 这两种应用各有优缺点,对于嵌入式开发者来说,两者都必须掌握。 : V( S+ X! p3 r; i, V" k1 N
一、HTTP网页服务器5 u5 `6 t/ B; F& h: R7 y$ J8 Q
先说下HTTP网页服务器获取MJPG码流的代码,首先是OKMX8MP-C在开发板端建立TCP服务器: - int TCP_Server_Found(socklen_t* socket_found , char* ip , int port)' J3 u" v' p# s+ L9 [& m
- { j, {- a9 t H/ F! Z G; R
- struct sockaddr_in servaddr;
, h6 z& a- J4 q0 K6 a- f, A - socklen_t addrsize = sizeof(struct sockaddr);
5 l9 X2 w5 g# C. D - bzero(&servaddr , sizeof(servaddr));
; ]% v$ l. L* r3 q - servaddr.sin_family = AF_INET;
* T1 K; C! m& ?& e3 J* } - servaddr.sin_addr.s_addr = inet_addr(ip);! e( u8 C( y4 @
- servaddr.sin_port = htons(port);
3 n- a6 r: |0 O7 E$ W - int ret;7 A- {) I' n7 ], D4 q1 {0 b) a
- IF( (*socket_found = socket(AF_INET , SOCK_STREAM , 0)) == -1). Y5 j1 Y0 s/ l
- {- U' k9 K6 {. [, l
- printf("Create socket error: %s (errno :%d)\n",strerror(errno),errno);
* a8 b; m+ r) ?* X$ H6 p c9 X" ?) z5 u - return -1;. `7 n, _3 h5 q B( z3 F: i
- }
* |4 p1 X+ ? W0 {8 B4 C z7 u - int on = 1;( D/ p) y$ y" f9 R: R6 B0 f
- if(setsockopt(*socket_found , SOL_SOCKET, SO_REUSEADDR, &on, sizeof(on)) < 0)8 Q4 ]# {- B0 J( M2 w
- {
' y7 k/ a6 N4 d1 y - printf("setsockopt error\n");0 T; D; H1 u5 Y3 Z2 Z7 I+ @
- }
" K& M/ s1 T! S3 Y2 } - ret = bind(*socket_found , (struct sockaddr *)&servaddr , addrsize);: V; y" W+ H( F4 ^; o, U
- if(ret == -1)7 ]4 R5 j/ b3 n! s5 y
- {+ c) R- }: p6 ]" _9 |( b
- printf("Tcp bind faiLED!\n");8 J9 K1 B0 A0 l4 v
- return -1;" n/ Z7 }9 P) R, p0 Z
- }* f: B. o; D3 J7 h$ W0 I
- if(listen(*socket_found , 5) == -1)
" a' Q& m: I' l2 v; n - {2 m% @3 q2 `( R* c: D
- printf("Listen failed!\n");
3 ?) o0 m: k5 m - return -1;! H9 Z. h) h1 g5 P8 C- |1 b3 U
- }
0 E e6 u; n5 Z% N4 O5 k - return 0;
! S1 z6 E8 `1 x4 Y$ F! f - }
复制代码其中setsockopt()函数是可选的,一般只用于规避socket()函数的建立错误。 建立了TCP服务器后,返回的socklen_t型实参在后面的HTTP网页服务器中需要用到。 HTTP网页服务器所属的TCP操作是需要另起轮询线程来让客户端进行accept()握手操作的,accept()之前的listen()倒是只需要执行一次即可,accept()握手操作和recv()接收操作需要创建一个死循环线程: - pthread_create(&tid_tcp_web_recv , NULL , Thread_TCP_Web_Recv , NULL);$ w |; k9 Y7 s# v5 c2 V
- void * Thread_TCP_Web_Recv(void *arg)8 S+ q. r9 [* a- |) ]1 ?8 a& C
- {
% C6 Y* ^! l* L% C - 。。。" e9 S0 M% {* | I# S/ r
- while(1)
% _0 ~" ]" M' E - {
T" a, P X/ n - fd_socket_conn = accept(socket_web_server , (struct sockaddr *)&sockaddr_in_conn , &addrsize);1 A9 e$ P% \/ ?% M6 G$ \5 g
- printf("fd_socket_conn = accept()\n");
6 o) u$ \8 C6 D+ B - 。。。, O3 k9 b# }1 g S4 ^" g0 ?
- recv(fd_socket_conn , recvbuf , 1000 , 0);. p# ~ y4 C. r
- }* R# h5 y" v5 q+ v& s4 p7 d
- 。。。
9 [' S9 Q, z. R+ G8 k - }
复制代码MJPG帧可以使用Grab操作获取,获取到的MJPG帧需要在TCP线程中读,在Grab操作线程中写,这种被多个线程访问的资源需要加锁防止读写冲突,即资源被Grab操作写入时,需要上锁,不允许其它线程访问,操作完成时需要解锁,允许其它线程访问: - pthread_mutex_lock(&pmt);
* E4 \* `: k$ ? - pic_tmpbuffer = pic.tmpbuffer;+ E3 e! y. d& |
- pic.tmpbytesused = buff.bytesused;
4 ]; u9 U* @! R' x% D+ K+ c - pic_tmpbytesused = pic.tmpbytesused;
) n. E9 o! o& ^) S& {% ~ - pthread_cond_broadcast(&pct);% E5 g0 X, i1 x8 I$ J
- pthread_mutex_unlock(&pmt);
复制代码线程互斥锁使用之前需要初始化: - pthread_mutex_t pmt;2 |9 Y5 R. L- C' I
- pthread_cond_t pct;
. M$ ^6 S2 E9 Z& ^ - int main(int argc, char* argv[])
& q# q0 }2 b, {! {2 ?. X, z( y - {
2 e) J. ]' n H) U. q& q: ? - ...
4 G& ?5 }) Z' d+ E' L% ~ - TCP_Server_Found(&socket_web_server , (char*)argv[2] , PORT_TCP); i( Q: Y H3 r/ ~
- pthread_mutex_init(&pmt , NULL);
- T4 f% {' N) a - pthread_create(&tid_tcp_web_recv , NULL , Thread_TCP_Web_Recv , NULL);$ v! |! J. f1 R E# \. [ r
- pthread_create(&tid_tcp_web_send , NULL , Thread_TCP_Web_Send , NULL);5 K2 N8 G6 o4 p7 J0 h- G
- .../ \' m) ]4 c' V# {( P0 u4 |# `
- while(1)6 \& w1 A" {- l: M. d
- {
/ e5 z$ _/ E% z/ a' w I: U - V4l2_Grab_Mjpeg(false , MJPEG_FILE_NAME);7 q- c( x( h6 h' V
- ...! V: U3 V+ H4 s! `* S* @
- }% ]" u2 {; A- q/ E: F! V: K- Y$ |
- ...
! h6 {& Z' m! S2 S - }
复制代码然后是发送的细节,发送图片文件之前,需要先发送HTTP标准头,这个相当于给发送图片或者其它类型的流数据铺路: - <p style="box-sizing: border-box; border: 0px; vertical-align: baseline; line-height: 26px;"><span style="text-indent: 32px;">
+ o) q# J. Z A( \+ L0 D8 H - </span></p><p style="box-sizing: border-box; border: 0px; vertical-align: baseline; line-height: 26px;"></p>
复制代码- #define STD_HEADER "Connection: close\r\n" \* B6 h" k* p' O' _* [
- "Server: MJPG-Streamer/0.2\r\n" \
0 R9 O7 P! z6 r7 L, E5 _2 N7 q: j - "Cache-Control: no-store, no-cache, must-revalidate, pre-check=0, post-check=0, max-age=0\r\n" \
) ^9 u1 E9 ], ?5 n - "Pragma: no-cache\r\n" \
5 X9 I( X- x- k4 ? - "Expires: Mon, 3 Jan 2000 12:34:56 GMT\r\n"% K, N# b) Q' x3 a; \8 E
- #define BOUNDARY "boundarydonotcross"2 Z9 x/ n) s8 m+ O
- printf("preparing header\n");0 w1 d! F: [: p* Y
- sprintf(buffer, "HTTP/1.0 200 OK\r\n" \$ J& T0 V" H; Z4 m
- "Access-Control-Allow-Origin: *\r\n" \
% _) W5 S# b* [& i0 G - STD_HEADER \
( H8 b; V: ~9 p - "Content-Type: multipart/x-mixed-replace;boundary=" BOUNDARY "\r\n" \
+ H' ?+ I9 @4 s - "\r\n" \0 g7 d; T0 K: P& s6 }
- "--" BOUNDARY "\r\n");* m5 R9 l( ^- m5 q- W& _- y2 C
- if(write(fd, buffer, strlen(buffer)) < 0)+ K0 b7 R) r1 d- N" E
- {! t) B% \# e3 ~" ~! k* U
- free(frame);
& w% C& G# _+ U0 J4 k, R - return;
' a2 y1 p% ~' ^1 L - }
复制代码发送完HTTP标准头之后,就需要发送内容头(Content-Type),这处的Content-Type为image/jpeg,同样,HTTP标准协议里面image支持的类型远不止jpeg一种,发送完内容头之后就是正文和boundary结尾,这样帧完整的HTTP头发送到指定的TCP GET地址,就会在浏览器中显示刚刚发送的图片: - <pre class="prettyprint lang-cpp" style="box-sizing: border-box; font-family: Monaco, Menlo, Consolas, "Courier New", monospace; font-size: 16px; white-space: pre-wrap; line-height: 1.38462; color: rgb(51, 51, 51); word-break: break-all; background-color: rgb(245, 245, 245); border: 0px; border-radius: 4px; vertical-align: baseline;"> sprintf(buffer, "Content-Type: image/jpeg\r\n" \
& x6 v3 j% ?8 u0 O( _- ? - "Content-Length: %d\r\n" \
& R. \" X2 D/ n7 c3 X* Y D9 S - "X-Timestamp: %d.%06d\r\n" \
9 i6 ~5 d# \1 }+ x& I - "\r\n", frame_size, (int)timestamp.tv_sec, (int)timestamp.tv_usec);) Z+ U8 u; s2 m6 K; `8 j
- printf("sending intemdiate header\n");
% h- M2 G I8 B# W r% r7 x, N - if(write(fd, buffer, strlen(buffer)) < 0)& q# m% B! V7 Z1 j6 N3 ?
- break;
5 h" M* Y+ M. x - printf("sending frame\n");" C7 k# Q: O$ s- b5 J- z4 B, O: `
- if(write(fd, frame, frame_size) < 0)
* U p6 U8 h2 d6 d - break;
* |( E; Q" c. ~! J - printf("sending boundary\n");
/ b3 ?3 O7 I; @% B( O+ _ - sprintf(buffer, "\r\n--" BOUNDARY "\r\n");. e" C T/ Y( d1 u
- if(write(fd, buffer, strlen(buffer)) < 0)% j4 [; q% _! \0 Y) x
- break;</pre><p style="box-sizing: border-box; border: 0px; font-size: 16px; vertical-align: baseline; line-height: 26px; color: rgb(51, 51, 51); font-family: " helvetica="" neue",="" helvetica,="" tahoma,="" arial,="" "microsoft="" yahei",="" "hiragino="" sans="" gb",="" "wenquanyi="" micro="" hei",="" sans-serif;"=""></p>
复制代码另外需要说明的是,TCP服务器线程在发送MJPEG流的时候是死循环发送的,因此TCP客户端在发送完GET指令之后,就会收到TCP服务器循环发送的图像缓存,TCP客户端会陷入忙等待状态无法再对外发送任何GET或者POST指令,从客户端使用者角度来看的效果就是网页一直在等待。  6 ~# b3 R3 m1 I' f

) @5 R# R2 u2 V; d二、UDP上位机UDP发送操作,同样需要先建立UDP Socket:
( Q$ d$ Q5 d4 G6 Z+ D0 m0 ?- int UDP_Send_Found(socklen_t* socket_found , struct sockaddr_in *addr , char* ip , int port)
1 D$ E- n$ p* F - {
- M2 V! j5 N6 i$ p( F - *socket_found = socket(AF_INET, SOCK_DGRAM, 0);
* ~8 ?/ T1 w' @5 U$ `/ z - if(*socket_found == (~0))" F8 {! u0 J8 {) n& a. ?! ]& K
- {
% ~( c9 i: ^8 } - printf("Create udp send socket failed!\n");
5 V# Z, _% V; D9 J7 u - return -1;
1 c* }* _. a- u" U' Q4 F% {2 X - }7 B/ O, L# @5 y; V* c4 I
- addr->sin_family = AF_INET;5 O) p( z1 z! R6 [
- addr->sin_addr.s_addr = inet_addr(ip);
! ]- m" T( t: {2 ~ S( x - addr->sin_port = htons(port);
5 a: D# l, J5 s7 K! S - memset(addr->sin_zero, 0, 8);
5 ~3 I* N8 M: L% v - return 0;3 d$ u/ {7 Q; B+ g" h# {7 f- l
- }
复制代码
7 a0 |0 X9 ^/ X4 f x1 p& Z
4 Q& L) b( O ~, w) s0 L; L1 u而UDP文件发送则要比HTTP发送简单得多,只需要将文件切片,每一片为固定长度的UDP帧长度,逐帧发送即可:
. S' D+ e! Q7 F; z
; D/ }$ K7 w& @9 s+ X3 _$ j9 D. K# W J ~& \1 t, y
- while(fend > 0)3 I3 s+ X+ h6 y J
- {
' n! `% t. ^* }( ^ - memset(picture.data , 0 , sizeof(picture.data));- B0 f @/ k" D/ r$ C
- fread(picture.data , UDP_FRAME_LEN , 1, fp);- C: o1 p; m; e6 H! K3 a/ E
- if(fend >= UDP_FRAME_LEN)# M% M0 q* e2 i8 r
- {5 D2 r* q; Y0 L! G5 X* b7 Y% Y: X
- picture.length = UDP_FRAME_LEN;$ h i: a# E7 R Z& w
- picture.fin = 0;
0 F9 R! M0 f3 u5 f' w - }8 W/ y) o7 l P5 `! i/ y& K- j
- else
$ r% K% j6 x# p7 V - {
3 Y% f' Z( r" g9 K2 W+ p - picture.length = fend;; @% p4 O3 K! e9 G$ u0 |
- picture.fin = 1;
/ f4 _/ l+ W0 v# p( l - }
2 [/ T( p" g4 i! ` - //printf("sendbytes = %d \n",sendbytes);4 C. Y) T' f5 ^4 { c7 T
- sendbytes = sendto(socket_send, (char *)&picture, sizeof(struct Package), 0, (struct sockaddr*)&addr,addr_len);5 ?% I( Z! p& ]# u( i d7 C
- if(sendbytes == -1). ]: l3 Q! r3 G2 \1 E8 @3 \
- {
' d3 o A! f5 b) ~' m, S4 o - printf("Send Picture Failed!d\n");0 |/ M9 A% R' a/ v) u
- return -1;
$ W) Q9 S7 J+ e/ U5 h! A7 i s - }9 e* U1 `% S. K, ?) _0 k' Q6 e- V
- else( e' T) s4 V( q0 I6 J- p6 A; S
- {
/ _9 m6 f( g: t$ V5 B - fend -= UDP_FRAME_LEN;
" l/ g( w7 _- u* O - }
* z, G! N, ?0 E# R/ J' L - }
复制代码 # N1 n/ Y3 X) G
& L! V2 {" K, x
$ |- Y% y- V) Y2 Q2 q6 X* ]( G, s' d/ l7 j6 L d/ ]% r
iMX8MPlus 核心板: https://www.forlinx.com/product/136.html |