本帖最后由 飞凌-marketing 于 2022-1-27 11:00 编辑
7 K% @! V$ \3 i8 C# z, S
9 O5 j1 i. ~- R2 K D+ _) r/ } 6 @# p: q! J+ G1 }/ ?
作者|donatello1996 来源 | 电子发烧友 题图|飞凌嵌入式 iMX8MPlus 核心板: https://www.forlinx.com/product/136.html- {! K* o0 P: J1 }
; ^) e8 X1 y9 t8 A本文采用的硬件板卡为飞凌嵌入式OKMX8MP-C开发板,系统版本Linux5.4.70+Qt5.15.0,主要介绍基于HTTP网页服务器和UDP上位机的MJPG码流传输。 MJPG格式作为一种持续传输的视频码流,在远程监控领域中应用较广,而实现这种远程监控的第三方应用最常见的有两种:浏览器HTTP网页、UDP上位机。
% \" R5 p$ ^" G$ `0 {7 I' H ( M# q9 u2 ~( ]- }# K2 Y
两者各有优势,对比鲜明,其中: 这两种应用各有优缺点,对于嵌入式开发者来说,两者都必须掌握。
4 i. t, |7 a8 F5 {! C( t8 @一、HTTP网页服务器4 ^4 Z/ @3 I8 Q, Z7 D* Z4 ~4 }
先说下HTTP网页服务器获取MJPG码流的代码,首先是OKMX8MP-C在开发板端建立TCP服务器: - int TCP_Server_Found(socklen_t* socket_found , char* ip , int port)
* @9 w s4 F. j% \, O; } - {; q5 r3 n: F, d5 b4 P1 q$ w
- struct sockaddr_in servaddr;# I% W6 V2 ^2 O5 g; x$ S }9 ?
- socklen_t addrsize = sizeof(struct sockaddr);
. ^' _/ a l& W6 h( f9 |' a - bzero(&servaddr , sizeof(servaddr));: Q' c* o( @9 E
- servaddr.sin_family = AF_INET;; A$ @" q* P6 V N- V. n
- servaddr.sin_addr.s_addr = inet_addr(ip);! S& a& z; H$ o( h7 `- y1 Z
- servaddr.sin_port = htons(port);
* J8 B% d: F1 [4 h: A# E6 C - int ret;
# W, Z" j- Y% d* q& n# V* e* Q- g3 o - IF( (*socket_found = socket(AF_INET , SOCK_STREAM , 0)) == -1)
7 {8 l4 B& P+ N( l0 H! a - {
9 y' G3 C0 f, Z& L* \ - printf("Create socket error: %s (errno :%d)\n",strerror(errno),errno);; L$ H, v+ h/ j, D+ V
- return -1;$ M; ~7 U8 j+ k, ?+ q
- }+ l1 P: u8 c' a
- int on = 1;) Y+ e+ X# `8 y
- if(setsockopt(*socket_found , SOL_SOCKET, SO_REUSEADDR, &on, sizeof(on)) < 0)
- }" R' \# w' z' f, e. H; S - {
; P( V! a% S$ n! C/ G - printf("setsockopt error\n");7 }. W/ ^: k. n2 b) W9 s
- }
7 `5 }1 J; U3 @* |( q - ret = bind(*socket_found , (struct sockaddr *)&servaddr , addrsize);
9 y) T6 z* B; u9 m - if(ret == -1)
( u* S% W; a9 f; n - {
" Z; ^6 A6 j5 N - printf("Tcp bind faiLED!\n");
Q# E$ y& K6 h- N# f7 T - return -1;
9 S6 H0 @8 g/ r/ W' L8 z - }
8 ]& L) _% i2 l0 I" K - if(listen(*socket_found , 5) == -1); E: R' L, @+ o
- {
3 b, _6 q% Q2 u( s; s - printf("Listen failed!\n");
+ }: d8 F9 p* \; M: A; \8 s - return -1;9 P2 e* i: K/ O! S) _
- }/ C: W3 e8 Y# c, M- a
- return 0;0 h: f8 F) x- o1 `' j' y
- }
复制代码其中setsockopt()函数是可选的,一般只用于规避socket()函数的建立错误。 建立了TCP服务器后,返回的socklen_t型实参在后面的HTTP网页服务器中需要用到。 HTTP网页服务器所属的TCP操作是需要另起轮询线程来让客户端进行accept()握手操作的,accept()之前的listen()倒是只需要执行一次即可,accept()握手操作和recv()接收操作需要创建一个死循环线程: - pthread_create(&tid_tcp_web_recv , NULL , Thread_TCP_Web_Recv , NULL);
) G9 P6 \0 W0 \" ^" j5 Q - void * Thread_TCP_Web_Recv(void *arg)% }; Z) }$ `7 Q7 F3 l; L
- {
$ \/ t8 W7 a( D" Y; i+ t' C - 。。。
3 | F; A: M" q6 y8 y - while(1)
3 v2 v3 |- L7 M9 o - {8 Z* |0 P* Q# _, Q% D
- fd_socket_conn = accept(socket_web_server , (struct sockaddr *)&sockaddr_in_conn , &addrsize);# T7 g4 A! K$ L1 q" l5 I
- printf("fd_socket_conn = accept()\n");+ w5 p- j' I1 e+ \
- 。。。! v6 _6 a6 S/ X5 r& Y, g+ I
- recv(fd_socket_conn , recvbuf , 1000 , 0);4 X, t6 b, {9 y, w( F$ o6 C
- }
- u+ k0 S8 b ?( t% n! i/ y+ A& V4 T - 。。。7 i+ s% [' T* x, o* g$ ?; V$ }
- }
复制代码MJPG帧可以使用Grab操作获取,获取到的MJPG帧需要在TCP线程中读,在Grab操作线程中写,这种被多个线程访问的资源需要加锁防止读写冲突,即资源被Grab操作写入时,需要上锁,不允许其它线程访问,操作完成时需要解锁,允许其它线程访问: - pthread_mutex_lock(&pmt);$ L! T+ b- V$ {- I4 h
- pic_tmpbuffer = pic.tmpbuffer;- B8 L2 Q9 W% W4 e
- pic.tmpbytesused = buff.bytesused;
b7 k/ e2 X' {8 V$ J! r3 v - pic_tmpbytesused = pic.tmpbytesused;
8 v) Q. V8 t5 q1 S0 W - pthread_cond_broadcast(&pct);* i2 f6 M& x% [- s" B
- pthread_mutex_unlock(&pmt);
复制代码线程互斥锁使用之前需要初始化: - pthread_mutex_t pmt;
5 N/ @, R% E, K6 _4 V - pthread_cond_t pct;
' S7 D0 g3 H( E. ^3 U t5 e - int main(int argc, char* argv[])& _; Z2 h/ _& N
- {, s1 [' c9 F" ?1 g A) X
- ...* Y) A# _; X( Y5 y9 D6 y
- TCP_Server_Found(&socket_web_server , (char*)argv[2] , PORT_TCP);
2 |1 t9 ~3 m2 J - pthread_mutex_init(&pmt , NULL);
' ^& A; m( B4 s" `, t. @: Y - pthread_create(&tid_tcp_web_recv , NULL , Thread_TCP_Web_Recv , NULL);3 c/ d9 R& {( c f$ h
- pthread_create(&tid_tcp_web_send , NULL , Thread_TCP_Web_Send , NULL);
# N7 V/ Q) \, l) b2 n" u) h - ...
1 f0 W; I0 n, j2 v# j* |: U' u0 V - while(1)' O9 M; f# B) N% Z
- {* T; F/ f# q" a: }
- V4l2_Grab_Mjpeg(false , MJPEG_FILE_NAME);
8 I( F- Y9 O9 Y2 S - ...
7 [: o; o" L) p& T/ v3 H - }
* D2 N* N* N) M0 U* ]. f8 n - ...
" o* U" s: Q# z+ }( U4 ^8 k - }
复制代码然后是发送的细节,发送图片文件之前,需要先发送HTTP标准头,这个相当于给发送图片或者其它类型的流数据铺路: - <p style="box-sizing: border-box; border: 0px; vertical-align: baseline; line-height: 26px;"><span style="text-indent: 32px;">
% Y5 G* W( _" [ C - </span></p><p style="box-sizing: border-box; border: 0px; vertical-align: baseline; line-height: 26px;"></p>
复制代码- #define STD_HEADER "Connection: close\r\n" \& ?3 {2 n3 U) J6 f& r+ e) C& T) Q
- "Server: MJPG-Streamer/0.2\r\n" \% I8 M- n2 M+ m" v* W8 P/ o
- "Cache-Control: no-store, no-cache, must-revalidate, pre-check=0, post-check=0, max-age=0\r\n" \
5 C( c) R! v: o, Z( E; x - "Pragma: no-cache\r\n" \
6 r2 D. K0 z: h0 E1 }0 ^ - "Expires: Mon, 3 Jan 2000 12:34:56 GMT\r\n": u: `3 q1 \7 |! T1 r( a
- #define BOUNDARY "boundarydonotcross", K5 v/ c: I, j: `' C* c
- printf("preparing header\n");+ |7 z4 T$ e$ ^0 |% @0 a
- sprintf(buffer, "HTTP/1.0 200 OK\r\n" \( V t( p u5 t+ M" m- i. Q# I( `
- "Access-Control-Allow-Origin: *\r\n" \
+ O' O# [1 }! w% Q% L$ b# J' `, s6 j - STD_HEADER \- k( z( {/ C; d& a* a
- "Content-Type: multipart/x-mixed-replace;boundary=" BOUNDARY "\r\n" \9 `; L7 m8 M. ]* `7 R' Y: |
- "\r\n" \0 e9 D. u. v' |! Q: U
- "--" BOUNDARY "\r\n");
) [/ O: C7 o" U3 ?5 a' _ - if(write(fd, buffer, strlen(buffer)) < 0)
+ l! R0 I4 a5 u+ R+ l% a$ _ - {
) ~* f) ?8 c6 n3 Y4 U - free(frame);1 G. i* k/ `( `' w/ V
- return;8 b& T: W. q3 o! Y
- }
复制代码发送完HTTP标准头之后,就需要发送内容头(Content-Type),这处的Content-Type为image/jpeg,同样,HTTP标准协议里面image支持的类型远不止jpeg一种,发送完内容头之后就是正文和boundary结尾,这样帧完整的HTTP头发送到指定的TCP GET地址,就会在浏览器中显示刚刚发送的图片: - <pre class="prettyprint lang-cpp" style="box-sizing: border-box; font-family: Monaco, Menlo, Consolas, "Courier New", monospace; font-size: 16px; white-space: pre-wrap; line-height: 1.38462; color: rgb(51, 51, 51); word-break: break-all; background-color: rgb(245, 245, 245); border: 0px; border-radius: 4px; vertical-align: baseline;"> sprintf(buffer, "Content-Type: image/jpeg\r\n" \
! d+ d& J, |* r% D x - "Content-Length: %d\r\n" \" }# _! C- g/ v) z1 C* `( z, P6 N
- "X-Timestamp: %d.%06d\r\n" \
( O/ Y! r% n% X( o' o$ f+ s - "\r\n", frame_size, (int)timestamp.tv_sec, (int)timestamp.tv_usec);
7 R8 U t: e- t, a - printf("sending intemdiate header\n");) e# x* i2 q( H
- if(write(fd, buffer, strlen(buffer)) < 0)! y* {7 P) v. S) f0 R8 Q. i
- break;- ~& U3 o: y/ Z Y5 E) U+ e! y
- printf("sending frame\n");; O! K, s/ ~9 _' a
- if(write(fd, frame, frame_size) < 0)
/ K/ |2 q& g$ a7 P1 @) Y - break;
' C. y5 ^, Y' H2 I$ O - printf("sending boundary\n");
# t2 N2 j) }. G/ [6 ~% ~ - sprintf(buffer, "\r\n--" BOUNDARY "\r\n");9 b; k% O5 q% s. C
- if(write(fd, buffer, strlen(buffer)) < 0)! f! ?# w% o, N, Q
- break;</pre><p style="box-sizing: border-box; border: 0px; font-size: 16px; vertical-align: baseline; line-height: 26px; color: rgb(51, 51, 51); font-family: " helvetica="" neue",="" helvetica,="" tahoma,="" arial,="" "microsoft="" yahei",="" "hiragino="" sans="" gb",="" "wenquanyi="" micro="" hei",="" sans-serif;"=""></p>
复制代码另外需要说明的是,TCP服务器线程在发送MJPEG流的时候是死循环发送的,因此TCP客户端在发送完GET指令之后,就会收到TCP服务器循环发送的图像缓存,TCP客户端会陷入忙等待状态无法再对外发送任何GET或者POST指令,从客户端使用者角度来看的效果就是网页一直在等待。 
( S! `2 p! R# h* S! h/ e
" n9 ~; N: }4 |' d |: z! G二、UDP上位机UDP发送操作,同样需要先建立UDP Socket:, z- A7 c! u, E4 C8 ?2 e7 p
- int UDP_Send_Found(socklen_t* socket_found , struct sockaddr_in *addr , char* ip , int port)
& [ Z2 o \0 R# l1 _5 R - {
) P/ u6 V( L; y% C: y' K# L# J! [ - *socket_found = socket(AF_INET, SOCK_DGRAM, 0);% R9 _- W, ?% [8 e0 n' b
- if(*socket_found == (~0))0 ^ g( B: `- F
- {
$ i0 c: g: P" C: c3 }6 i1 `* E3 H - printf("Create udp send socket failed!\n");6 _1 b( c2 o, h$ D7 M1 e! l
- return -1;$ S1 V0 x' q2 L1 z. Z
- }
; v+ r0 J2 Y) ? M4 r4 M3 _ - addr->sin_family = AF_INET;+ m% |; e2 R! u& b( r! w6 u- r
- addr->sin_addr.s_addr = inet_addr(ip);/ ~: S8 F' i) G. ~5 n
- addr->sin_port = htons(port);
) \5 @( y' p5 |% g - memset(addr->sin_zero, 0, 8);! l$ V9 ~& {9 h
- return 0;
4 n! G3 Y, @7 d( q - }
复制代码 l# U6 W% z9 O+ i I# Z
" C9 j$ o; [2 A1 L; @( G4 p1 }
而UDP文件发送则要比HTTP发送简单得多,只需要将文件切片,每一片为固定长度的UDP帧长度,逐帧发送即可:) u6 i2 Y) b' i; ^& I
3 h' Q( ]3 y" S) s! d; c
4 S( |2 k, v7 d# }! _. S N6 f- while(fend > 0)
$ r$ c! } v, V5 o; G; e+ f( N7 H - {0 k. V9 x: C* {& J: Q
- memset(picture.data , 0 , sizeof(picture.data));9 S( N) F9 X; T* W* Q
- fread(picture.data , UDP_FRAME_LEN , 1, fp);
! S% ~& {2 V2 D5 A) C - if(fend >= UDP_FRAME_LEN)- `" a+ l8 K( b! T
- {' p2 _0 `0 i/ Q4 ?6 G1 u
- picture.length = UDP_FRAME_LEN;
/ {8 S( [9 ^( \" f6 ~' u2 B. Z - picture.fin = 0;! Y, j1 l4 f- }: U" e7 ?
- }
! ] T4 W. f* R) D+ ?4 } - else, @ E* W- T0 q) N# H3 w
- {
1 ^) x9 m3 W$ v" y, j: B - picture.length = fend;. ?( S" e+ t1 [
- picture.fin = 1;
- X8 e$ L. l8 j* e: O! @% [. | - }
3 z7 X! Y7 |+ b/ N - //printf("sendbytes = %d \n",sendbytes);
+ }' `$ I" j5 L# N) @# } - sendbytes = sendto(socket_send, (char *)&picture, sizeof(struct Package), 0, (struct sockaddr*)&addr,addr_len);
. ^' T7 c: ^# x- b) _6 {# e3 g! p5 d: y - if(sendbytes == -1)
6 V8 d% q8 ]) s% \7 g - {
& _0 N9 g: u6 [2 F5 B2 k - printf("Send Picture Failed!d\n");
' C, j' b6 J8 D. n# h$ f/ ` - return -1;
( Y$ p9 Y: I8 l9 `% @: o - }4 F& u( M. n4 ?' o S) V; m; \9 ]( V
- else
$ s+ b- Q! Q' X1 \ - {
% A9 f5 R; i) v% C1 R - fend -= UDP_FRAME_LEN;0 `3 @; B& ~2 z
- }
7 K: N4 n! X( B' A' y - }
复制代码 4 g5 V/ D( [2 u! ?; k( P9 y* `" U+ W" i Z
, z- E- `& Y2 I ; U2 {1 `) I3 D- q% O0 `
6 G& p. E5 ]- ^
iMX8MPlus 核心板: https://www.forlinx.com/product/136.html |