本帖最后由 飞凌-marketing 于 2022-1-27 11:00 编辑
/ Y4 n, |- M% a4 G4 @+ M3 p/ r& r1 e% M$ A% {1 E7 W$ O: {2 ~
 1 Y4 `" m9 t: ^( ^4 q6 c, b W
作者|donatello1996 来源 | 电子发烧友 题图|飞凌嵌入式 iMX8MPlus 核心板: https://www.forlinx.com/product/136.html
4 Y4 V1 z/ ?0 X$ r8 k: n' y& K \9 E5 F
# M7 y: Q, v# ^! z" g+ p本文采用的硬件板卡为飞凌嵌入式OKMX8MP-C开发板,系统版本Linux5.4.70+Qt5.15.0,主要介绍基于HTTP网页服务器和UDP上位机的MJPG码流传输。 MJPG格式作为一种持续传输的视频码流,在远程监控领域中应用较广,而实现这种远程监控的第三方应用最常见的有两种:浏览器HTTP网页、UDP上位机。
5 o& D" Q3 \" p% n ) C8 z2 ^% h$ A/ p# B$ `# x6 j, c' D
两者各有优势,对比鲜明,其中: 这两种应用各有优缺点,对于嵌入式开发者来说,两者都必须掌握。
! N5 y1 L: Z1 }" U一、HTTP网页服务器4 p6 o6 D1 K& C. a
先说下HTTP网页服务器获取MJPG码流的代码,首先是OKMX8MP-C在开发板端建立TCP服务器: - int TCP_Server_Found(socklen_t* socket_found , char* ip , int port)
( l4 L/ e6 M+ [5 p - {
0 E; L6 ]2 n; ?- F8 _0 ^! p - struct sockaddr_in servaddr;# f: V* F* L& ?6 f2 ?/ I/ ~
- socklen_t addrsize = sizeof(struct sockaddr);
4 r( K, ]" R) m - bzero(&servaddr , sizeof(servaddr));8 [, V; g! r0 s4 {4 p7 x
- servaddr.sin_family = AF_INET;
) Q) [" s$ Z. |" r - servaddr.sin_addr.s_addr = inet_addr(ip);: w B5 D( w# w% Q* S
- servaddr.sin_port = htons(port);( _5 V+ T* f7 ?7 K; U
- int ret;
' ?* k" Q* E g: o - IF( (*socket_found = socket(AF_INET , SOCK_STREAM , 0)) == -1)9 R! a& S: w: Q/ z. a% M
- {
0 e) q8 ]2 I: q' G7 @; {7 x - printf("Create socket error: %s (errno :%d)\n",strerror(errno),errno);9 z3 I+ Z+ S9 y0 a! x; r
- return -1;
4 v0 V- j6 G2 J% R( w - }. W4 c9 R/ Y* a0 ] T: P+ p1 d
- int on = 1;
3 ? k. H! F7 l- M - if(setsockopt(*socket_found , SOL_SOCKET, SO_REUSEADDR, &on, sizeof(on)) < 0)' M" q5 r& W& y) P
- {
6 A% v$ h* a7 M! L" X* d8 e8 r - printf("setsockopt error\n");
' p9 o5 m. F E2 y' d) |$ U - }
) z5 O& U) v% G" C" _! N1 s - ret = bind(*socket_found , (struct sockaddr *)&servaddr , addrsize);* _- y9 V& Y" L& y- r1 s N8 S
- if(ret == -1). B- Y1 l2 M2 ~; f
- {: d. }4 b" Y) q( x# E; X# _: u" R7 u
- printf("Tcp bind faiLED!\n");
" m, n) d) K" G - return -1;- }, }. I) W0 C
- }
( l1 B' f7 N( p1 ]) v3 S% U - if(listen(*socket_found , 5) == -1)
8 T$ {. S9 n) \5 q( |: _ - {, j: v6 `6 N! W) v
- printf("Listen failed!\n");; Y& ]' N2 z! f% d# z! \
- return -1;* h8 l A9 w" K! Z$ ^ t
- }
) w7 `6 Q4 m j# s - return 0;8 ?# ~- F) [2 P$ Z' z$ \
- }
复制代码其中setsockopt()函数是可选的,一般只用于规避socket()函数的建立错误。 建立了TCP服务器后,返回的socklen_t型实参在后面的HTTP网页服务器中需要用到。 HTTP网页服务器所属的TCP操作是需要另起轮询线程来让客户端进行accept()握手操作的,accept()之前的listen()倒是只需要执行一次即可,accept()握手操作和recv()接收操作需要创建一个死循环线程: - pthread_create(&tid_tcp_web_recv , NULL , Thread_TCP_Web_Recv , NULL);2 O5 |9 A: ?; N ~+ T" Y
- void * Thread_TCP_Web_Recv(void *arg)
4 v3 I: A# \8 I2 e! T - {5 P6 A& s [# K! _) y
- 。。。9 R+ J; B9 x5 W$ S
- while(1), w& p1 |. J R
- {, y* X" F8 t& u9 q* \% [9 z0 y
- fd_socket_conn = accept(socket_web_server , (struct sockaddr *)&sockaddr_in_conn , &addrsize);
. ^$ ] x" {- `+ t - printf("fd_socket_conn = accept()\n");4 h4 F/ m( _$ c
- 。。。& z0 w& I% H* d
- recv(fd_socket_conn , recvbuf , 1000 , 0);. Z$ S3 S" e( c) ]3 Q
- }
2 ^7 O: E- T* m1 i/ q - 。。。+ Z0 A- G+ P7 l
- }
复制代码MJPG帧可以使用Grab操作获取,获取到的MJPG帧需要在TCP线程中读,在Grab操作线程中写,这种被多个线程访问的资源需要加锁防止读写冲突,即资源被Grab操作写入时,需要上锁,不允许其它线程访问,操作完成时需要解锁,允许其它线程访问: - pthread_mutex_lock(&pmt);
8 e/ A) C/ Q& v - pic_tmpbuffer = pic.tmpbuffer;
% P* K8 n. N" n; I - pic.tmpbytesused = buff.bytesused;
5 A. t$ y0 }# [4 v5 |& M! e - pic_tmpbytesused = pic.tmpbytesused;
( ]# m! Y$ x k4 b% w0 | - pthread_cond_broadcast(&pct);
8 z* | {9 }$ ~ - pthread_mutex_unlock(&pmt);
复制代码线程互斥锁使用之前需要初始化: - pthread_mutex_t pmt;" I" F( \8 ]7 ^' [+ a7 x
- pthread_cond_t pct;
. i# R" S" W% {0 h- s6 ~ - int main(int argc, char* argv[])+ D: l/ J! s$ T: A
- {$ M# z& X g* p, C, V
- ...7 t3 P, W6 e- Z' h4 ]# I
- TCP_Server_Found(&socket_web_server , (char*)argv[2] , PORT_TCP);
* h b# U: M+ U - pthread_mutex_init(&pmt , NULL);1 v, ^0 g1 a4 w) H+ O# L8 y
- pthread_create(&tid_tcp_web_recv , NULL , Thread_TCP_Web_Recv , NULL);6 H# m. Z$ d2 j$ [+ h
- pthread_create(&tid_tcp_web_send , NULL , Thread_TCP_Web_Send , NULL);2 ^. [/ N/ A, _: d; E9 F
- ..., ~) k# {' H# o" s
- while(1)
4 ~( ^( B. M2 s1 }) f6 G+ [9 M/ D - {
! k: |; f4 a7 `* K4 q# i# }+ ] - V4l2_Grab_Mjpeg(false , MJPEG_FILE_NAME);8 Y+ w% H& P) a8 V8 f( c' r
- ...
$ _, c- ]6 _/ M) F. K& q# S$ g - } i. k5 n4 m* n9 B* }5 a t
- ...
% D; t* p+ _- H5 M$ J# x6 {+ z - }
复制代码然后是发送的细节,发送图片文件之前,需要先发送HTTP标准头,这个相当于给发送图片或者其它类型的流数据铺路: - <p style="box-sizing: border-box; border: 0px; vertical-align: baseline; line-height: 26px;"><span style="text-indent: 32px;">+ E8 W7 o! z: n ^3 Y
- </span></p><p style="box-sizing: border-box; border: 0px; vertical-align: baseline; line-height: 26px;"></p>
复制代码- #define STD_HEADER "Connection: close\r\n" \7 b8 _ I2 U/ P, B
- "Server: MJPG-Streamer/0.2\r\n" \7 `1 p @! k5 A
- "Cache-Control: no-store, no-cache, must-revalidate, pre-check=0, post-check=0, max-age=0\r\n" \* ~! D% ~& s; ~) H, e7 q& |! K* ?
- "Pragma: no-cache\r\n" \
' p4 b( v" H& G; U, [ - "Expires: Mon, 3 Jan 2000 12:34:56 GMT\r\n"/ v/ }! J1 z0 N5 @4 r
- #define BOUNDARY "boundarydonotcross"
. ]5 d' d* W( n! ] - printf("preparing header\n");4 u/ o7 n3 `4 o7 Q
- sprintf(buffer, "HTTP/1.0 200 OK\r\n" \8 w3 c# M- ]- r1 e' r' Z e
- "Access-Control-Allow-Origin: *\r\n" \
- |- V9 V$ z! c - STD_HEADER \7 E& h2 h8 g6 a$ i; n0 {
- "Content-Type: multipart/x-mixed-replace;boundary=" BOUNDARY "\r\n" \
9 e, J- u9 X4 h# r - "\r\n" \
4 l7 H. q' z4 X, t$ _: z - "--" BOUNDARY "\r\n");# r6 F; i2 {+ i& a- E0 t! M
- if(write(fd, buffer, strlen(buffer)) < 0), T: G* ^0 z/ ^$ n4 s+ s% v
- {) c# A% G/ @% Z2 F$ L% x0 w. }
- free(frame);; V4 {' {( Z: G, q* M8 x& b
- return;, b0 E l0 G2 N: T; M! r- [
- }
复制代码发送完HTTP标准头之后,就需要发送内容头(Content-Type),这处的Content-Type为image/jpeg,同样,HTTP标准协议里面image支持的类型远不止jpeg一种,发送完内容头之后就是正文和boundary结尾,这样帧完整的HTTP头发送到指定的TCP GET地址,就会在浏览器中显示刚刚发送的图片: - <pre class="prettyprint lang-cpp" style="box-sizing: border-box; font-family: Monaco, Menlo, Consolas, "Courier New", monospace; font-size: 16px; white-space: pre-wrap; line-height: 1.38462; color: rgb(51, 51, 51); word-break: break-all; background-color: rgb(245, 245, 245); border: 0px; border-radius: 4px; vertical-align: baseline;"> sprintf(buffer, "Content-Type: image/jpeg\r\n" \; j) X1 e! t' s3 l
- "Content-Length: %d\r\n" \
z/ J# V; R( \- z - "X-Timestamp: %d.%06d\r\n" \' l4 C: ^# A g, b0 T9 k! ~+ r
- "\r\n", frame_size, (int)timestamp.tv_sec, (int)timestamp.tv_usec);
" i* e1 Z( r# a5 G, S) m - printf("sending intemdiate header\n");
0 F/ m; y2 W7 F3 |/ ~2 S+ P# |; u - if(write(fd, buffer, strlen(buffer)) < 0)
" J8 l6 o! j1 U6 y - break;
/ |9 W' U* a' e) X: v$ y% F - printf("sending frame\n");" M" r E1 x6 M# U& \, w
- if(write(fd, frame, frame_size) < 0)
5 p: \, @+ }$ r - break;+ S' y3 v0 h( S @( B Q, O# f
- printf("sending boundary\n");
- s8 R, `& {) c1 S4 a - sprintf(buffer, "\r\n--" BOUNDARY "\r\n");" m. l9 @$ g( t% [, s/ t X5 l
- if(write(fd, buffer, strlen(buffer)) < 0)
( Z% p, ~) m* S, S( z! I* O J9 t - break;</pre><p style="box-sizing: border-box; border: 0px; font-size: 16px; vertical-align: baseline; line-height: 26px; color: rgb(51, 51, 51); font-family: " helvetica="" neue",="" helvetica,="" tahoma,="" arial,="" "microsoft="" yahei",="" "hiragino="" sans="" gb",="" "wenquanyi="" micro="" hei",="" sans-serif;"=""></p>
复制代码另外需要说明的是,TCP服务器线程在发送MJPEG流的时候是死循环发送的,因此TCP客户端在发送完GET指令之后,就会收到TCP服务器循环发送的图像缓存,TCP客户端会陷入忙等待状态无法再对外发送任何GET或者POST指令,从客户端使用者角度来看的效果就是网页一直在等待。  : ]6 K/ }# N, [7 T
 + H) [$ G% G! G; ^! E% x1 D( p
二、UDP上位机UDP发送操作,同样需要先建立UDP Socket:3 U9 ?$ l/ I; V9 E' P& `
- int UDP_Send_Found(socklen_t* socket_found , struct sockaddr_in *addr , char* ip , int port)
* `+ s8 X4 _) b3 G - {
, c5 @/ G& o% M2 P5 F8 C; t8 d - *socket_found = socket(AF_INET, SOCK_DGRAM, 0);
) u( x' P! P- Y. [; O+ K+ X9 ~9 ` - if(*socket_found == (~0))
! R1 C) z/ ]+ l& r2 ? - {
. e( s! D8 a' }; M0 X$ N Q - printf("Create udp send socket failed!\n");# N" J& E2 R4 I. X/ u: u2 K
- return -1;
+ J, j( R7 v* d! g - }
# A; D1 b1 t* P) F: I - addr->sin_family = AF_INET;
! S8 a) k& T, X* U7 w7 O - addr->sin_addr.s_addr = inet_addr(ip);
8 Y" u0 h* b4 P5 {: s/ G+ a0 F - addr->sin_port = htons(port);( _: m) D1 a* [
- memset(addr->sin_zero, 0, 8);
+ N: S6 ], h- b - return 0;+ d `7 W, t. V2 f# X) Z$ b" L
- }
复制代码
: q `+ G' u' e3 q* y
3 j) N1 K7 z; K5 g. w0 ^: P9 X, j而UDP文件发送则要比HTTP发送简单得多,只需要将文件切片,每一片为固定长度的UDP帧长度,逐帧发送即可:
! l. U6 S1 A0 c4 ^
" j8 o/ x0 Z. [0 p- b/ c* p, T5 f, M! ^8 M6 f
- while(fend > 0) h7 l6 B1 t: i# |
- {
: r0 P+ j% \. x# P5 {/ @ - memset(picture.data , 0 , sizeof(picture.data));
% D- B% k& T8 {7 K+ q0 ? - fread(picture.data , UDP_FRAME_LEN , 1, fp);5 v6 _ ]) K1 Z8 f
- if(fend >= UDP_FRAME_LEN)* L% K2 x% t! E8 _) f
- {
% I' a/ M/ H0 F, A& O/ i - picture.length = UDP_FRAME_LEN;7 q5 n) @) t3 k# B+ B
- picture.fin = 0;
' H% H2 g b6 t& N. f6 B4 M5 J - }- L( ?/ w+ T1 m# w
- else; c+ ]; h+ e4 z8 I: R
- {0 y" W. l0 e I4 x0 |( h
- picture.length = fend;1 i( {: _) G3 }3 Q* p
- picture.fin = 1;
0 Z, j! o9 Y8 ~" [8 d - }0 r! y/ w2 O, F% q( i# \: f
- //printf("sendbytes = %d \n",sendbytes);
5 C1 u5 ], J- U$ Y# [6 Y - sendbytes = sendto(socket_send, (char *)&picture, sizeof(struct Package), 0, (struct sockaddr*)&addr,addr_len);
: C! M( ^ |( m6 U0 e, P3 n. k - if(sendbytes == -1)5 I' y$ H3 c2 [
- {
) t; r$ h" s0 \ - printf("Send Picture Failed!d\n");
7 h c" ~: H( @. O' y" t$ \ - return -1;
5 Z* u) i4 K O' m: f# N - }
4 q' V) W) [" l1 ^% q. j _0 d - else6 |9 r2 D" w* _& ~, Q
- {
: W' o5 W9 ]! J! h L - fend -= UDP_FRAME_LEN;
( m& E% V; S( y, ?, { - }2 }/ C+ r. S4 W1 { t. |
- }
复制代码 * U) A) h2 ~3 {8 }1 G4 e
' N# k0 w9 M8 T' s) M. s

" i, x# e6 ]" s7 ?3 Y: v, N! t
# r3 I0 ]) x- I1 u: U8 U! ~9 ~iMX8MPlus 核心板: https://www.forlinx.com/product/136.html |