本帖最后由 飞凌-marketing 于 2022-1-27 11:00 编辑
+ M: L+ `( o1 C: C# n- W$ C" p
% l7 @7 E9 r: k7 U! P% h8 q' J 7 l" v& u& j' ?: v' K
作者|donatello1996 来源 | 电子发烧友 题图|飞凌嵌入式 iMX8MPlus 核心板: https://www.forlinx.com/product/136.html
' U( }2 p: I- s4 S* \4 I* {
/ R% P. E* H7 O* `本文采用的硬件板卡为飞凌嵌入式OKMX8MP-C开发板,系统版本Linux5.4.70+Qt5.15.0,主要介绍基于HTTP网页服务器和UDP上位机的MJPG码流传输。 MJPG格式作为一种持续传输的视频码流,在远程监控领域中应用较广,而实现这种远程监控的第三方应用最常见的有两种:浏览器HTTP网页、UDP上位机。
3 U5 Y7 N6 S" P. N0 s" V
8 N1 z1 M- C- H( U两者各有优势,对比鲜明,其中: 这两种应用各有优缺点,对于嵌入式开发者来说,两者都必须掌握。 ( E4 ^) [" f6 x( `
一、HTTP网页服务器
( Z1 C' f" v* x6 j3 {先说下HTTP网页服务器获取MJPG码流的代码,首先是OKMX8MP-C在开发板端建立TCP服务器: - int TCP_Server_Found(socklen_t* socket_found , char* ip , int port)
9 Z/ t) w/ J0 l! n - {
) S9 [1 Z* ^! L5 M - struct sockaddr_in servaddr;5 J$ K, r l$ }- Q! O
- socklen_t addrsize = sizeof(struct sockaddr);
; W+ T4 G5 u$ q) B7 {/ p5 A - bzero(&servaddr , sizeof(servaddr));5 a2 O* M- v+ j/ o
- servaddr.sin_family = AF_INET;
+ t9 |% X' P0 n* x$ ~5 W - servaddr.sin_addr.s_addr = inet_addr(ip);0 _' ]! `5 m! b
- servaddr.sin_port = htons(port);+ s4 s6 e8 p2 `) V
- int ret;+ t3 R* C% ^/ N! U. F, d8 b/ o
- IF( (*socket_found = socket(AF_INET , SOCK_STREAM , 0)) == -1)4 G1 H) p R& G! i! I9 b; _8 I
- {
- I, ~" u @+ x# n& @, p, H - printf("Create socket error: %s (errno :%d)\n",strerror(errno),errno);
1 {/ @% Y; I' n' ]5 R) j - return -1;
0 U. e3 {8 b2 |$ x - }, Y6 j5 n E# e6 f
- int on = 1;
5 g, f) k" \% c S' \6 \ - if(setsockopt(*socket_found , SOL_SOCKET, SO_REUSEADDR, &on, sizeof(on)) < 0)
6 s+ u. b1 U5 M0 M6 f3 X( h4 M - {
& S& x' K! x2 _: e0 t4 i1 { - printf("setsockopt error\n");( G: @1 ^# Q" A* U" H
- }
% X8 m5 s* s1 ^) l4 |& g7 F - ret = bind(*socket_found , (struct sockaddr *)&servaddr , addrsize);
+ x, I* p4 E$ Q2 K3 K - if(ret == -1)6 q1 |" }: H& c( R, z$ s/ U
- {
7 p& u- v& D$ O7 ~, `' Z1 N0 x, x - printf("Tcp bind faiLED!\n");* P$ @! S: b4 Q' l7 l( q6 A
- return -1;
9 a( V$ L. b$ }# F. f& }: J - }/ _* x2 K' M3 c& X! b2 W0 P
- if(listen(*socket_found , 5) == -1)
) W" _9 A8 X3 \0 f* o% _4 a2 A - {
5 j. [4 j1 p" i T# O - printf("Listen failed!\n");% X+ t9 R n( q7 o2 Y% n1 ?
- return -1;$ c7 [% `( b' Z. U5 o' Z3 e( @
- }
2 n4 A4 r3 R4 Z( x- L# O - return 0;! {. u" S& [0 w p u2 \
- }
复制代码其中setsockopt()函数是可选的,一般只用于规避socket()函数的建立错误。 建立了TCP服务器后,返回的socklen_t型实参在后面的HTTP网页服务器中需要用到。 HTTP网页服务器所属的TCP操作是需要另起轮询线程来让客户端进行accept()握手操作的,accept()之前的listen()倒是只需要执行一次即可,accept()握手操作和recv()接收操作需要创建一个死循环线程: - pthread_create(&tid_tcp_web_recv , NULL , Thread_TCP_Web_Recv , NULL);
$ J( X+ M, R& w6 J) {! M - void * Thread_TCP_Web_Recv(void *arg)
! W) y+ A# K* A. c - {
2 J/ }+ \: r5 {# l4 p' N - 。。。5 @$ B( ^. z2 }; F( a1 j; N
- while(1)
& o: o; R' t. ?& J/ h - {: c. d: s' ?1 V
- fd_socket_conn = accept(socket_web_server , (struct sockaddr *)&sockaddr_in_conn , &addrsize);: F* G3 [, S% B# c& y4 r; d6 b
- printf("fd_socket_conn = accept()\n");- `; y$ K! Y& q$ | b
- 。。。9 R7 l5 ?1 i9 H# k3 S% E9 r7 Q. P( _
- recv(fd_socket_conn , recvbuf , 1000 , 0);
7 d0 N8 F/ A* `1 [' {- a, ]9 J2 [ - }( y# X( X; \7 w9 R* K) ]2 X
- 。。。
- I( m, F) D H: z - }
复制代码MJPG帧可以使用Grab操作获取,获取到的MJPG帧需要在TCP线程中读,在Grab操作线程中写,这种被多个线程访问的资源需要加锁防止读写冲突,即资源被Grab操作写入时,需要上锁,不允许其它线程访问,操作完成时需要解锁,允许其它线程访问: - pthread_mutex_lock(&pmt);: ^8 T( w, E m/ D2 r; W; w! h
- pic_tmpbuffer = pic.tmpbuffer;
6 Q. k9 e* S; A - pic.tmpbytesused = buff.bytesused;5 B& q/ t* U0 ^2 M- x" s
- pic_tmpbytesused = pic.tmpbytesused;
! N8 y/ G* D7 o( { c - pthread_cond_broadcast(&pct);$ }) f- H" z3 b' L# s
- pthread_mutex_unlock(&pmt);
复制代码线程互斥锁使用之前需要初始化: - pthread_mutex_t pmt;+ c" P: V& O) H9 w6 `' b; c6 ~
- pthread_cond_t pct;9 A9 w2 w/ R H4 g$ E2 N; _
- int main(int argc, char* argv[])
7 B8 y! _! O, n% E/ `: k - {7 Z: L3 P" Q6 w8 a
- ...
9 e8 `7 A0 X0 ^( X. l, w* i - TCP_Server_Found(&socket_web_server , (char*)argv[2] , PORT_TCP);' }9 }* K$ v2 N+ s0 V+ \1 @
- pthread_mutex_init(&pmt , NULL);
; L. ^* j {8 z& i- ^ - pthread_create(&tid_tcp_web_recv , NULL , Thread_TCP_Web_Recv , NULL);
+ ?: F/ c+ _0 K7 ? - pthread_create(&tid_tcp_web_send , NULL , Thread_TCP_Web_Send , NULL);4 ?2 k" |( ]/ t; {
- ...6 e" \2 h! {+ |
- while(1)
- q! h+ ~: A: x0 ^: B/ a - {0 E' O4 l1 q0 f
- V4l2_Grab_Mjpeg(false , MJPEG_FILE_NAME);
+ }& k5 ]7 T/ l. G& S - ...
9 G! ]1 w; a4 R. U+ d. A - }
1 y# |/ S, l# X t+ a# t: { - ...4 V0 U5 t$ T! Z/ ]' h' `: B) w$ S
- }
复制代码然后是发送的细节,发送图片文件之前,需要先发送HTTP标准头,这个相当于给发送图片或者其它类型的流数据铺路: - <p style="box-sizing: border-box; border: 0px; vertical-align: baseline; line-height: 26px;"><span style="text-indent: 32px;">8 `0 X& l5 y: A
- </span></p><p style="box-sizing: border-box; border: 0px; vertical-align: baseline; line-height: 26px;"></p>
复制代码- #define STD_HEADER "Connection: close\r\n" \
7 k4 A4 j- p! Z0 I3 A - "Server: MJPG-Streamer/0.2\r\n" \
; M4 B8 z, B- x" Y - "Cache-Control: no-store, no-cache, must-revalidate, pre-check=0, post-check=0, max-age=0\r\n" \8 T! u" X5 C8 Y1 J, o
- "Pragma: no-cache\r\n" \8 v9 d; u6 J) d% l2 ?( ]
- "Expires: Mon, 3 Jan 2000 12:34:56 GMT\r\n"1 G) A% P4 \' }: s
- #define BOUNDARY "boundarydonotcross"
- m# k5 @: t' a% @7 t8 @ - printf("preparing header\n");2 W; L! [7 \" A- \9 n
- sprintf(buffer, "HTTP/1.0 200 OK\r\n" \1 W+ a t" d8 o
- "Access-Control-Allow-Origin: *\r\n" \2 n$ A' R+ w: N
- STD_HEADER \
) S, T7 S" @6 d4 ` - "Content-Type: multipart/x-mixed-replace;boundary=" BOUNDARY "\r\n" \
7 W4 u6 l5 O+ S3 y( W7 X - "\r\n" \6 L0 e4 U. x" J
- "--" BOUNDARY "\r\n");
; y' G& J0 {5 ~) M. M; {" S - if(write(fd, buffer, strlen(buffer)) < 0)
- ^0 h9 B4 k! {6 w. B0 ?" W) A3 Y - {
' H5 o& z8 U( O \: o - free(frame);
; \7 N" F: A' j+ x$ O/ L5 R - return;
/ r- u' M \1 X9 s: M - }
复制代码发送完HTTP标准头之后,就需要发送内容头(Content-Type),这处的Content-Type为image/jpeg,同样,HTTP标准协议里面image支持的类型远不止jpeg一种,发送完内容头之后就是正文和boundary结尾,这样帧完整的HTTP头发送到指定的TCP GET地址,就会在浏览器中显示刚刚发送的图片: - <pre class="prettyprint lang-cpp" style="box-sizing: border-box; font-family: Monaco, Menlo, Consolas, "Courier New", monospace; font-size: 16px; white-space: pre-wrap; line-height: 1.38462; color: rgb(51, 51, 51); word-break: break-all; background-color: rgb(245, 245, 245); border: 0px; border-radius: 4px; vertical-align: baseline;"> sprintf(buffer, "Content-Type: image/jpeg\r\n" \4 b6 n: m) K6 |/ E% y) a' M; @3 O
- "Content-Length: %d\r\n" \8 Y5 a+ p: q# |8 Z' G" _" ?
- "X-Timestamp: %d.%06d\r\n" \# h1 Z' a1 I+ w" S& R- Q
- "\r\n", frame_size, (int)timestamp.tv_sec, (int)timestamp.tv_usec);
$ a3 n2 [0 l0 J5 A E1 h - printf("sending intemdiate header\n");
7 H5 z) ?1 [" R/ k- C - if(write(fd, buffer, strlen(buffer)) < 0)
" x0 [/ {) o! s( |. N7 I6 }* K - break;
8 L/ d# R9 |; L( D/ A2 s; ?; _ - printf("sending frame\n");
3 U3 n" u: R; N0 a9 P' |* S+ P# U - if(write(fd, frame, frame_size) < 0)
e9 L5 `) C* Q' ]! k* ^% \" e - break;
( d* }8 H0 |. d# u8 C - printf("sending boundary\n");6 @" p- L Y3 W4 b" `( o0 N5 S
- sprintf(buffer, "\r\n--" BOUNDARY "\r\n"); ~1 o" G. m- b& C9 l
- if(write(fd, buffer, strlen(buffer)) < 0)
. r: a, t! m3 g% h6 C; O9 g - break;</pre><p style="box-sizing: border-box; border: 0px; font-size: 16px; vertical-align: baseline; line-height: 26px; color: rgb(51, 51, 51); font-family: " helvetica="" neue",="" helvetica,="" tahoma,="" arial,="" "microsoft="" yahei",="" "hiragino="" sans="" gb",="" "wenquanyi="" micro="" hei",="" sans-serif;"=""></p>
复制代码另外需要说明的是,TCP服务器线程在发送MJPEG流的时候是死循环发送的,因此TCP客户端在发送完GET指令之后,就会收到TCP服务器循环发送的图像缓存,TCP客户端会陷入忙等待状态无法再对外发送任何GET或者POST指令,从客户端使用者角度来看的效果就是网页一直在等待。  7 U1 B" t! L, {
 3 q! U7 s3 j$ k" g& t+ k$ r& V
二、UDP上位机UDP发送操作,同样需要先建立UDP Socket:
0 i7 y' o( V2 H7 t7 P- int UDP_Send_Found(socklen_t* socket_found , struct sockaddr_in *addr , char* ip , int port)$ n* `7 @! ^$ h3 O; ~ S$ N
- {
) @8 B, O- |! J& D# ]6 C - *socket_found = socket(AF_INET, SOCK_DGRAM, 0);
4 A4 e5 w( t- J - if(*socket_found == (~0))
8 y- l3 T6 F' c - {
6 [! h+ x- k' p - printf("Create udp send socket failed!\n");; S1 P' B$ E4 c; Z: c: D
- return -1;. k" `0 M' V" T0 B
- }
! P" y7 e& {$ @, M7 R: ^1 w) J - addr->sin_family = AF_INET;- M5 R, O5 K2 ]
- addr->sin_addr.s_addr = inet_addr(ip);, {- p# q5 Z: N
- addr->sin_port = htons(port);
% A1 P+ Z7 ~, i) U2 e2 { - memset(addr->sin_zero, 0, 8);- V2 {7 [, i b% T$ K
- return 0;# {1 I0 d. f7 j8 B# a
- }
复制代码 & Z. H# q0 h& v6 b+ Y; ~& \
6 S4 S9 L4 b0 M) ?# _: A
而UDP文件发送则要比HTTP发送简单得多,只需要将文件切片,每一片为固定长度的UDP帧长度,逐帧发送即可:5 E5 n: N6 g& f. p) ?* U
. Y% \1 j \1 _3 a" k# ^7 l3 r
4 V3 Q6 N8 e3 f- while(fend > 0)7 `- f8 c `, i# w
- {: L0 O9 N* Q7 h, B6 g, L. Q) q l9 o
- memset(picture.data , 0 , sizeof(picture.data));
1 C) i' U& I' g) {3 B. ~% r - fread(picture.data , UDP_FRAME_LEN , 1, fp);
8 {+ Z( B0 j) [! K2 ^; I - if(fend >= UDP_FRAME_LEN)* a3 E. n3 E* t
- {# m a* L; v8 V+ i% x) u+ d
- picture.length = UDP_FRAME_LEN;
% J3 @! T$ S8 D* n I - picture.fin = 0;
" A y9 ?7 K3 V1 y1 R5 }+ ]: b+ ^& D - }' l1 m3 n3 w6 y! U7 ]9 S
- else3 J* `9 f2 D" e3 f
- {
$ K) g( r0 k4 A - picture.length = fend;
) H) A3 F, G+ ]" m w1 F - picture.fin = 1;
, A) x! [" G3 L5 u - }
O6 W+ t, b" g - //printf("sendbytes = %d \n",sendbytes);
& s: i% t) o# s6 V$ F/ q - sendbytes = sendto(socket_send, (char *)&picture, sizeof(struct Package), 0, (struct sockaddr*)&addr,addr_len);
5 J" J/ n) A$ M - if(sendbytes == -1)
) L b v$ K4 M$ p9 A1 W9 X - {
2 H* f" ^+ z! L% a9 c - printf("Send Picture Failed!d\n");* J6 }1 w/ E7 Z3 U( l' Y9 x
- return -1;
- s1 X4 M7 f0 x7 {: E' Z - }
9 A( u! k( a: ~( \. e3 w+ u - else
N) s/ O- q5 B8 {2 m - {; X# h. G' n$ L- S- r! v# O" J
- fend -= UDP_FRAME_LEN;
8 }* r5 `& k, G - }
4 t' `! ?5 p5 q7 b6 \1 X - }
复制代码
3 a1 }0 j( a8 T- w. n7 P* O: Q- J( Q; y' Q. n" I2 s3 |. f

8 r% ~ Y4 O. { ^# Y p) @5 Q' q; q) V% L
iMX8MPlus 核心板: https://www.forlinx.com/product/136.html |