本帖最后由 飞凌-marketing 于 2022-1-27 11:00 编辑
' B9 `; O/ p7 A/ c2 Y, O
7 f Y; w/ \5 o& P6 L4 @5 x
, W* d- V, D9 E' L; ?% `. m3 @& I, W2 C作者|donatello1996 来源 | 电子发烧友 题图|飞凌嵌入式 iMX8MPlus 核心板: https://www.forlinx.com/product/136.html5 {; y0 R: y& N# e1 P- S
2 C5 C. T9 D& b% Q
本文采用的硬件板卡为飞凌嵌入式OKMX8MP-C开发板,系统版本Linux5.4.70+Qt5.15.0,主要介绍基于HTTP网页服务器和UDP上位机的MJPG码流传输。 MJPG格式作为一种持续传输的视频码流,在远程监控领域中应用较广,而实现这种远程监控的第三方应用最常见的有两种:浏览器HTTP网页、UDP上位机。
- j( b' e7 i' J( M 4 x: W2 R$ S! }! b
两者各有优势,对比鲜明,其中: 这两种应用各有优缺点,对于嵌入式开发者来说,两者都必须掌握。
0 Z# r" k9 k- X0 |4 r8 Y0 D一、HTTP网页服务器5 G- B% ?5 v( t! H2 M5 ^. F5 W; W
先说下HTTP网页服务器获取MJPG码流的代码,首先是OKMX8MP-C在开发板端建立TCP服务器: - int TCP_Server_Found(socklen_t* socket_found , char* ip , int port)& {) ]; Z3 \: O0 `; U7 J, ~
- {
: S' b1 ~1 d2 E* }6 Q% Z1 I3 k# \6 c - struct sockaddr_in servaddr;5 V( x, ?1 r5 X. s$ \. p
- socklen_t addrsize = sizeof(struct sockaddr);6 y$ s5 y5 x7 t% Q
- bzero(&servaddr , sizeof(servaddr));
1 V, j) r; ]$ d2 q& z5 V0 T" X6 a8 Y - servaddr.sin_family = AF_INET;# A, S( O) r4 v4 A) q6 A6 s) z
- servaddr.sin_addr.s_addr = inet_addr(ip);
0 x4 I0 F* B% x" A: V7 f - servaddr.sin_port = htons(port);
0 I0 Q. Q9 B3 y# L - int ret;
$ J, r1 c7 S; |$ L$ i - IF( (*socket_found = socket(AF_INET , SOCK_STREAM , 0)) == -1)
- L% Y8 B- O* i6 m' c/ q9 l5 B - {1 B) l9 L F3 x
- printf("Create socket error: %s (errno :%d)\n",strerror(errno),errno);7 ], `5 s- @% E6 L1 I
- return -1;
8 o* }4 D8 i" }" {: d - }. B2 s9 {2 Y& S- b$ M
- int on = 1;
0 |" Y& v! v$ I: S - if(setsockopt(*socket_found , SOL_SOCKET, SO_REUSEADDR, &on, sizeof(on)) < 0)1 b+ Y$ R) m" D1 A- M
- {
9 {) R2 h2 N, _ - printf("setsockopt error\n");
* _# H( _+ s8 |, Y - }
) i, u/ D2 x! ?6 f9 k9 } - ret = bind(*socket_found , (struct sockaddr *)&servaddr , addrsize);
. b0 J$ d7 W m* \' \ - if(ret == -1)
& D" e+ L& n% m' e - {
" y2 a8 Q5 T5 P# v - printf("Tcp bind faiLED!\n");
3 Q! \ s1 e* t - return -1;
1 L+ k5 M" l% h+ J - }
~, M) j: h2 }6 B5 O - if(listen(*socket_found , 5) == -1)
8 {# z: b: V2 O' o, |9 }/ V/ g - {: p3 x" n4 C) y; e' N A
- printf("Listen failed!\n");
+ {4 }7 R! s6 M5 J. r0 n7 v( o1 _ - return -1;
o3 d# D, r6 _: N7 g1 K9 R, Q - }- s: @5 d. i! n1 J' T. d2 z
- return 0;8 l" T+ P& [, ^$ e; ]* f
- }
复制代码其中setsockopt()函数是可选的,一般只用于规避socket()函数的建立错误。 建立了TCP服务器后,返回的socklen_t型实参在后面的HTTP网页服务器中需要用到。 HTTP网页服务器所属的TCP操作是需要另起轮询线程来让客户端进行accept()握手操作的,accept()之前的listen()倒是只需要执行一次即可,accept()握手操作和recv()接收操作需要创建一个死循环线程: - pthread_create(&tid_tcp_web_recv , NULL , Thread_TCP_Web_Recv , NULL);7 \% s( Q, V- n I
- void * Thread_TCP_Web_Recv(void *arg)
4 e2 O& c5 u/ ^0 |3 W - {* `+ K' f% G: _9 f* U
- 。。。1 v: I0 [8 p% [: z4 r
- while(1)
! ], b3 t% b B `3 K/ G8 J) q2 G& J* ~ - { Z( l5 }8 b1 u% }. h3 Q
- fd_socket_conn = accept(socket_web_server , (struct sockaddr *)&sockaddr_in_conn , &addrsize);
. x. Z+ w2 y7 B( [& K2 l# i W, _ - printf("fd_socket_conn = accept()\n");
3 b0 _$ ]/ N# g g - 。。。
7 d0 E% D% e; {- Q5 |6 ^; ? P* ]+ E - recv(fd_socket_conn , recvbuf , 1000 , 0);
, [) b* A7 `" G" l. m1 T' { - }% _5 x/ h, D% t+ l1 C
- 。。。* ~( M' v- U* E7 u2 G
- }
复制代码MJPG帧可以使用Grab操作获取,获取到的MJPG帧需要在TCP线程中读,在Grab操作线程中写,这种被多个线程访问的资源需要加锁防止读写冲突,即资源被Grab操作写入时,需要上锁,不允许其它线程访问,操作完成时需要解锁,允许其它线程访问: - pthread_mutex_lock(&pmt);
0 x% X3 v9 O5 B9 J1 X0 B - pic_tmpbuffer = pic.tmpbuffer;
! j. a- R4 Z% D- G& ]5 [& r - pic.tmpbytesused = buff.bytesused;- c4 Q7 P/ P/ q
- pic_tmpbytesused = pic.tmpbytesused;
$ A6 g( n2 L4 i8 j9 n u" H- Q - pthread_cond_broadcast(&pct);
`/ |; d' w- k0 m) Q+ b - pthread_mutex_unlock(&pmt);
复制代码线程互斥锁使用之前需要初始化: - pthread_mutex_t pmt;+ C$ Z# w6 m3 ?* _& f
- pthread_cond_t pct;+ v2 F5 a' w9 O
- int main(int argc, char* argv[])2 }; L. P( r7 D6 z! O9 e# f
- {% x( G9 a) k& E8 Z$ Y. I5 {8 d' o3 g
- ...
, m! @: V9 b+ o0 O - TCP_Server_Found(&socket_web_server , (char*)argv[2] , PORT_TCP);
' y- z5 Q& T( W" m7 m& J) x - pthread_mutex_init(&pmt , NULL);
% q7 U: w/ t. p. z9 M - pthread_create(&tid_tcp_web_recv , NULL , Thread_TCP_Web_Recv , NULL);7 {/ G8 ]3 L* x" ^8 L/ ?+ h7 w5 Z$ A
- pthread_create(&tid_tcp_web_send , NULL , Thread_TCP_Web_Send , NULL);7 l3 e9 o J D
- ...! P6 P: v: }( ?
- while(1)
1 V$ y( G1 q8 y9 y8 ^ - {
& Z! o- P, }8 T. Z4 G - V4l2_Grab_Mjpeg(false , MJPEG_FILE_NAME);
& j; e" j+ D9 _& f - ...
, v& x8 j) ?$ v F, ]5 W - }
) y5 m5 _# u/ E _; a h0 ? - ...6 J% R% T2 }, F6 c) b, v( Z
- }
复制代码然后是发送的细节,发送图片文件之前,需要先发送HTTP标准头,这个相当于给发送图片或者其它类型的流数据铺路: - <p style="box-sizing: border-box; border: 0px; vertical-align: baseline; line-height: 26px;"><span style="text-indent: 32px;">
! {: m5 P7 e* t; P - </span></p><p style="box-sizing: border-box; border: 0px; vertical-align: baseline; line-height: 26px;"></p>
复制代码- #define STD_HEADER "Connection: close\r\n" \- B7 K8 W- {- [1 q5 b* A
- "Server: MJPG-Streamer/0.2\r\n" \ s5 c( n5 x% G% u$ [8 m4 ^
- "Cache-Control: no-store, no-cache, must-revalidate, pre-check=0, post-check=0, max-age=0\r\n" \
& j+ U d2 m, Z+ e" |( r( L - "Pragma: no-cache\r\n" \+ k8 N; w/ i7 a* y3 {( j, @
- "Expires: Mon, 3 Jan 2000 12:34:56 GMT\r\n"9 G9 }$ L; k, d/ e
- #define BOUNDARY "boundarydonotcross"
, H- f! w: ~# f( v9 [ - printf("preparing header\n");' ~. s* G; x6 N9 Z- S# J1 F
- sprintf(buffer, "HTTP/1.0 200 OK\r\n" \ v! Q6 h" r" y3 i; ] s" B
- "Access-Control-Allow-Origin: *\r\n" \0 |% P1 Z5 i4 i" s+ e
- STD_HEADER \
! U# h% B& i3 e0 p+ @# }9 ]" k - "Content-Type: multipart/x-mixed-replace;boundary=" BOUNDARY "\r\n" \" c5 w) u5 K7 L' I
- "\r\n" \
4 N8 p7 c0 c! X/ F8 [2 g - "--" BOUNDARY "\r\n");+ S% A# E% d: W5 d- @
- if(write(fd, buffer, strlen(buffer)) < 0)
, H" b* G9 P: m* V) M; Y: r) s7 e - {
' A2 M+ f) C8 c/ U - free(frame);! M9 b$ m, S u/ _
- return;8 y1 O/ p0 z6 H* ?' u# U7 Q5 m/ u
- }
复制代码发送完HTTP标准头之后,就需要发送内容头(Content-Type),这处的Content-Type为image/jpeg,同样,HTTP标准协议里面image支持的类型远不止jpeg一种,发送完内容头之后就是正文和boundary结尾,这样帧完整的HTTP头发送到指定的TCP GET地址,就会在浏览器中显示刚刚发送的图片: - <pre class="prettyprint lang-cpp" style="box-sizing: border-box; font-family: Monaco, Menlo, Consolas, "Courier New", monospace; font-size: 16px; white-space: pre-wrap; line-height: 1.38462; color: rgb(51, 51, 51); word-break: break-all; background-color: rgb(245, 245, 245); border: 0px; border-radius: 4px; vertical-align: baseline;"> sprintf(buffer, "Content-Type: image/jpeg\r\n" \7 o5 h, v9 h' }0 a; P" O; L6 h
- "Content-Length: %d\r\n" \3 y8 u5 C- x$ Q# J
- "X-Timestamp: %d.%06d\r\n" \
" o6 t3 b6 V: R W: H, @ - "\r\n", frame_size, (int)timestamp.tv_sec, (int)timestamp.tv_usec);
# @8 I, i5 _2 w: J* K: b1 t$ O - printf("sending intemdiate header\n");
7 E5 h: z w c+ c; @/ _ - if(write(fd, buffer, strlen(buffer)) < 0)
- a3 S# n& t% d8 H5 U3 F% ~% h - break;
$ @' I% k8 Z8 p; ` - printf("sending frame\n");7 G. p+ b4 b& f% i% i% c0 b/ a. V
- if(write(fd, frame, frame_size) < 0): Q# f2 U0 Z, J$ D( k$ G
- break;1 f' v% o* [/ `9 X/ H! V3 S# f* u
- printf("sending boundary\n");8 ~: u" V6 U$ m$ }' @; l" n
- sprintf(buffer, "\r\n--" BOUNDARY "\r\n");* F: K8 h* J! e$ N0 P( s3 g% E
- if(write(fd, buffer, strlen(buffer)) < 0)0 x3 h4 y$ T* [) A1 Z. s
- break;</pre><p style="box-sizing: border-box; border: 0px; font-size: 16px; vertical-align: baseline; line-height: 26px; color: rgb(51, 51, 51); font-family: " helvetica="" neue",="" helvetica,="" tahoma,="" arial,="" "microsoft="" yahei",="" "hiragino="" sans="" gb",="" "wenquanyi="" micro="" hei",="" sans-serif;"=""></p>
复制代码另外需要说明的是,TCP服务器线程在发送MJPEG流的时候是死循环发送的,因此TCP客户端在发送完GET指令之后,就会收到TCP服务器循环发送的图像缓存,TCP客户端会陷入忙等待状态无法再对外发送任何GET或者POST指令,从客户端使用者角度来看的效果就是网页一直在等待。  . g" t! _! x2 g& h
 $ [8 _! U. W$ [2 o
二、UDP上位机UDP发送操作,同样需要先建立UDP Socket:
6 o4 P) S1 h! v" T. c# g- int UDP_Send_Found(socklen_t* socket_found , struct sockaddr_in *addr , char* ip , int port)
4 M% e" f+ O3 h; n: \ - {; t, w( e7 N( [7 f
- *socket_found = socket(AF_INET, SOCK_DGRAM, 0);. Y9 L" i, ? n
- if(*socket_found == (~0))
& t0 }# T" p3 l c6 k/ { - {- J+ t/ g: h( T, g$ U/ I3 P! E
- printf("Create udp send socket failed!\n");) {' ~& ~9 i; C+ V8 q
- return -1;
! {$ z4 P' h, w# V' o - }
- v2 m, j r7 p, k U( T6 g# A5 h - addr->sin_family = AF_INET;6 _3 X$ K* U0 O( ]2 R% V/ g( l
- addr->sin_addr.s_addr = inet_addr(ip);% h+ S# ]4 S. i) P1 n
- addr->sin_port = htons(port);9 d4 X! ^# Q5 w0 V
- memset(addr->sin_zero, 0, 8);
4 o0 Z# h+ E( H4 k& m8 U - return 0;
, T5 s. w0 `! M0 S' c$ e' Q4 v - }
复制代码
1 S; \# `/ R* I2 Z* i; P* S+ o5 t+ Z5 H( U, S# A
而UDP文件发送则要比HTTP发送简单得多,只需要将文件切片,每一片为固定长度的UDP帧长度,逐帧发送即可:
# d* d! o: H2 F. h) X- t
! i& P; O# ?% Z! {1 Z6 ?$ j. g5 y* `; J
- while(fend > 0)* @$ V# o+ E2 t0 O. ]2 Z
- {0 j9 L" \/ A* q" r7 l4 y
- memset(picture.data , 0 , sizeof(picture.data));
9 a ?+ b# C4 ?* z( A' w- c* t - fread(picture.data , UDP_FRAME_LEN , 1, fp);+ i; @3 K) w. M8 g5 ~
- if(fend >= UDP_FRAME_LEN)
3 \% M7 d& X! O5 J7 ]9 r - {
3 q. ~( E2 g8 g! e4 Q' a - picture.length = UDP_FRAME_LEN;
+ k1 }" q1 p$ ^8 s% l( \1 Q: Y& Y! F6 } - picture.fin = 0;- `9 n8 d6 C; t2 i3 a/ `
- }5 \! H6 T' g7 B, w2 F) t) [
- else4 f9 q- @4 |7 L
- {
^( d- }1 H* Q% o% [2 i4 _' q+ D - picture.length = fend;' ^3 y) y* B0 C2 }+ s3 C
- picture.fin = 1;
( ]* m$ d! M9 a0 B2 x! O/ b - }: Z+ w2 L9 l! q
- //printf("sendbytes = %d \n",sendbytes);
! W/ V8 l% Q! t: F; l" n - sendbytes = sendto(socket_send, (char *)&picture, sizeof(struct Package), 0, (struct sockaddr*)&addr,addr_len);) v6 `; k+ P5 O2 f% z9 ?
- if(sendbytes == -1)
2 ^3 `% Q; _5 j4 g$ u6 e - {1 }0 Y' G1 W% h/ d5 r: F. @: t w
- printf("Send Picture Failed!d\n");
_$ v, ?$ Y& z. n9 @+ t - return -1;& u: C1 P: _, U
- }
$ [' x& u. w( G; I1 Z5 P5 s4 _: | - else! @0 s& o+ q; g1 }
- {- q4 l6 n7 f" i: T
- fend -= UDP_FRAME_LEN;3 e# h" F |: K/ |+ r7 x/ a
- }
+ j4 Q8 J: L: Z9 B4 C; T - }
复制代码
! y0 [7 U- s4 ~" u( D! J& _7 L
3 @9 j$ }" O1 M0 s1 a2 X- C 7 v$ F) X# M2 W, P. l# y$ m
" E* a$ q0 q4 F$ s# Z3 UiMX8MPlus 核心板: https://www.forlinx.com/product/136.html |