本帖最后由 飞凌-marketing 于 2022-1-27 11:00 编辑
( k, l- w% h8 g- @+ `% u$ A& p& f o0 k& u! k& |) d, Z

& G8 n6 T$ ~. E! g0 g8 S作者|donatello1996 来源 | 电子发烧友 题图|飞凌嵌入式 iMX8MPlus 核心板: https://www.forlinx.com/product/136.html
9 \7 g7 D$ ?3 p }" |( c" }" _$ C9 \$ n
本文采用的硬件板卡为飞凌嵌入式OKMX8MP-C开发板,系统版本Linux5.4.70+Qt5.15.0,主要介绍基于HTTP网页服务器和UDP上位机的MJPG码流传输。 MJPG格式作为一种持续传输的视频码流,在远程监控领域中应用较广,而实现这种远程监控的第三方应用最常见的有两种:浏览器HTTP网页、UDP上位机。 8 D+ `" C( e* ^8 S. M7 J

7 q- Q- B. l4 U' z两者各有优势,对比鲜明,其中: 这两种应用各有优缺点,对于嵌入式开发者来说,两者都必须掌握。 5 {1 Y7 L+ C- h- e e4 ~* \, d
一、HTTP网页服务器
$ f H! t9 i y8 w$ \8 F先说下HTTP网页服务器获取MJPG码流的代码,首先是OKMX8MP-C在开发板端建立TCP服务器: - int TCP_Server_Found(socklen_t* socket_found , char* ip , int port) B2 v! X8 f3 t! d: C
- {& _0 C$ u- K6 }! M& d5 H# c) s) L
- struct sockaddr_in servaddr;
/ w) {5 E% w4 w7 n, u+ a - socklen_t addrsize = sizeof(struct sockaddr);
% i4 ^' S* `6 l4 G& { - bzero(&servaddr , sizeof(servaddr));
3 a F% f, Z- ~& A - servaddr.sin_family = AF_INET;
, z5 N/ ~' z( @+ V, \ - servaddr.sin_addr.s_addr = inet_addr(ip);
" N4 \# q% X9 k9 `1 Q - servaddr.sin_port = htons(port);. J0 Z& c' G1 k6 L8 K" E
- int ret;
+ N: c+ \: I% a! G6 `3 r - IF( (*socket_found = socket(AF_INET , SOCK_STREAM , 0)) == -1)2 [0 p1 l% t o+ g
- {# W2 |+ w/ @# F+ c
- printf("Create socket error: %s (errno :%d)\n",strerror(errno),errno);6 b5 ~ j% m6 Q" }% }8 n
- return -1;
( @5 c, O4 I2 L6 H8 l7 [ - }
6 z, h& m% [: Z5 X) W1 W4 E ~ - int on = 1;
7 l' \# {" d; ?- _+ J/ M! m. ?' b - if(setsockopt(*socket_found , SOL_SOCKET, SO_REUSEADDR, &on, sizeof(on)) < 0)8 D# l) j0 E8 ^, m
- {2 R( L( l2 G6 n* f$ l
- printf("setsockopt error\n");+ o. _) f! H7 U9 m" a0 K1 S
- }
' M7 a$ D" r; Z4 K! z - ret = bind(*socket_found , (struct sockaddr *)&servaddr , addrsize);" D/ s; q: W7 U/ Z: @
- if(ret == -1)' j7 h8 O& y: @
- {# M. j: D3 d0 p
- printf("Tcp bind faiLED!\n");
5 `8 y& M) e2 G - return -1;) W( k1 l& e/ Z& m$ g
- }3 @' {+ ?* X7 ^0 c+ F$ i' H) q2 d" K q
- if(listen(*socket_found , 5) == -1)
: j/ O j7 B8 i6 f - {. i% b4 J# x' I0 e8 d
- printf("Listen failed!\n");
: a3 F4 H1 A# j2 _% C - return -1;
# S! X8 o; ^, y- {. @3 _0 v+ Z! K - }
, J% W- i$ E2 [' h - return 0;! S* j2 r; a5 U) a9 N. O
- }
复制代码其中setsockopt()函数是可选的,一般只用于规避socket()函数的建立错误。 建立了TCP服务器后,返回的socklen_t型实参在后面的HTTP网页服务器中需要用到。 HTTP网页服务器所属的TCP操作是需要另起轮询线程来让客户端进行accept()握手操作的,accept()之前的listen()倒是只需要执行一次即可,accept()握手操作和recv()接收操作需要创建一个死循环线程: - pthread_create(&tid_tcp_web_recv , NULL , Thread_TCP_Web_Recv , NULL);, K* X2 U- d- U P/ r* p# j
- void * Thread_TCP_Web_Recv(void *arg)
" y$ K8 ?+ j; _6 ? - {% X. n+ [# j, w- {' |* `
- 。。。
7 b& V5 I Z1 O - while(1)- n. @8 E; T0 i1 t
- {8 ]! e- X A/ J: T9 ? K$ t
- fd_socket_conn = accept(socket_web_server , (struct sockaddr *)&sockaddr_in_conn , &addrsize);' g C8 R2 d2 I' @* [4 [
- printf("fd_socket_conn = accept()\n");
) N q% S. q9 ` - 。。。
9 o$ c: M: H6 r G" L* N6 s - recv(fd_socket_conn , recvbuf , 1000 , 0);
: }2 S7 M- |- V, w: n- w+ x - }
h/ J; b. z& C1 v! r; u - 。。。
: \$ L4 P/ C: u) h - }
复制代码MJPG帧可以使用Grab操作获取,获取到的MJPG帧需要在TCP线程中读,在Grab操作线程中写,这种被多个线程访问的资源需要加锁防止读写冲突,即资源被Grab操作写入时,需要上锁,不允许其它线程访问,操作完成时需要解锁,允许其它线程访问: - pthread_mutex_lock(&pmt);
$ R# z# z( L* J3 L, F - pic_tmpbuffer = pic.tmpbuffer;
- G2 F. K0 B. ]9 o2 M3 P* } - pic.tmpbytesused = buff.bytesused;- K5 E$ F# F2 q% {, q r
- pic_tmpbytesused = pic.tmpbytesused;
* Y4 T2 c7 Q! `2 R# Q, ]1 N8 { - pthread_cond_broadcast(&pct);
5 M y, B$ G, d4 p2 h: r8 k9 X4 a& z) [6 v - pthread_mutex_unlock(&pmt);
复制代码线程互斥锁使用之前需要初始化: - pthread_mutex_t pmt;4 W' k" E, ?$ u. c0 I, W2 a
- pthread_cond_t pct; [; P: Q- d8 L( ~, s' L* R" q. H
- int main(int argc, char* argv[])
: Z4 J; T8 R% w/ e9 D* d& q - {
& M, h, }4 X( a6 n# Q; m0 v2 e/ A - ...0 w+ p3 Z$ c! E
- TCP_Server_Found(&socket_web_server , (char*)argv[2] , PORT_TCP);
, `. P2 F; j) R0 X8 D - pthread_mutex_init(&pmt , NULL);
2 Y) T2 Y7 |& P( ~ - pthread_create(&tid_tcp_web_recv , NULL , Thread_TCP_Web_Recv , NULL);
4 b4 m/ n/ f E1 c" J, R2 K - pthread_create(&tid_tcp_web_send , NULL , Thread_TCP_Web_Send , NULL);
. F: U1 Y1 u g! Z2 l# F - ...8 Y* Z; g; _: F5 E( W1 s, b1 T; I
- while(1)
/ j7 x2 e9 ^) m, y( g& o. o1 M - {
3 X5 e& V! I9 i. g$ _5 R - V4l2_Grab_Mjpeg(false , MJPEG_FILE_NAME);
. v g8 R& V+ U* [: H - ...* R$ R/ l% Q) `9 D5 ^" B3 n' J5 U
- }
0 {- C% N( @$ v% d - ...
4 c* m O) R1 @+ V9 _# O - }
复制代码然后是发送的细节,发送图片文件之前,需要先发送HTTP标准头,这个相当于给发送图片或者其它类型的流数据铺路: - <p style="box-sizing: border-box; border: 0px; vertical-align: baseline; line-height: 26px;"><span style="text-indent: 32px;">2 Z) O, i0 V4 i! f1 Z( B
- </span></p><p style="box-sizing: border-box; border: 0px; vertical-align: baseline; line-height: 26px;"></p>
复制代码- #define STD_HEADER "Connection: close\r\n" \' U) ^* l8 G8 w- P% {' [
- "Server: MJPG-Streamer/0.2\r\n" \
; P, N; f6 }# S0 U/ @3 P6 r: b - "Cache-Control: no-store, no-cache, must-revalidate, pre-check=0, post-check=0, max-age=0\r\n" \
8 |4 x! z& J/ @$ [! J1 P$ A - "Pragma: no-cache\r\n" \4 Y& Z- d# e0 l, Q
- "Expires: Mon, 3 Jan 2000 12:34:56 GMT\r\n"4 c, Y1 R4 L n/ m1 h+ @5 D) y- T
- #define BOUNDARY "boundarydonotcross"
1 D U1 c. v* W5 ~% u - printf("preparing header\n");
x/ b3 Y6 k" N: O2 @ - sprintf(buffer, "HTTP/1.0 200 OK\r\n" \
& V4 e$ o5 h' r% ~, e' T: z$ ] - "Access-Control-Allow-Origin: *\r\n" \* X# y$ B3 w; l8 O3 b, H4 L I% Z
- STD_HEADER \
# v, ~3 G( b# U) d' i% ?5 G - "Content-Type: multipart/x-mixed-replace;boundary=" BOUNDARY "\r\n" \3 H# P p' K% F* ?, q) u `! x8 N1 H$ H
- "\r\n" \" n; L6 f2 h8 {6 r+ E
- "--" BOUNDARY "\r\n");* r0 ^: s- x! C; {
- if(write(fd, buffer, strlen(buffer)) < 0)
, A3 W6 G1 ^2 n! ?1 _) F9 r3 y" Y5 B& t - {& l+ Z( x' U ^# a6 s
- free(frame);6 ?$ w9 ^+ E8 k* a: J
- return;! M k+ C# l9 t" `! X/ P# z2 C& G
- }
复制代码发送完HTTP标准头之后,就需要发送内容头(Content-Type),这处的Content-Type为image/jpeg,同样,HTTP标准协议里面image支持的类型远不止jpeg一种,发送完内容头之后就是正文和boundary结尾,这样帧完整的HTTP头发送到指定的TCP GET地址,就会在浏览器中显示刚刚发送的图片: - <pre class="prettyprint lang-cpp" style="box-sizing: border-box; font-family: Monaco, Menlo, Consolas, "Courier New", monospace; font-size: 16px; white-space: pre-wrap; line-height: 1.38462; color: rgb(51, 51, 51); word-break: break-all; background-color: rgb(245, 245, 245); border: 0px; border-radius: 4px; vertical-align: baseline;"> sprintf(buffer, "Content-Type: image/jpeg\r\n" \& ?8 W& L) }7 W6 T1 V# |! N6 l
- "Content-Length: %d\r\n" \; L! t2 v0 A% j, C9 z
- "X-Timestamp: %d.%06d\r\n" \
9 _8 A) ~, c0 L8 \9 t) e: F - "\r\n", frame_size, (int)timestamp.tv_sec, (int)timestamp.tv_usec);
6 u2 A3 L7 j( b) P6 Y; | L& Z - printf("sending intemdiate header\n");
- A7 g: w: G V4 [ - if(write(fd, buffer, strlen(buffer)) < 0)1 `, A# q" u5 w- S: e
- break;
9 i, D* w- Y* u - printf("sending frame\n");
( P1 O; w' n8 c, Z8 d; q6 ` - if(write(fd, frame, frame_size) < 0); p- s& k! S0 z
- break;3 K' F. h9 b1 \+ D$ b
- printf("sending boundary\n");
1 m8 c0 D! A% j- I# Z: y - sprintf(buffer, "\r\n--" BOUNDARY "\r\n");2 \% Q S$ Y; m" P' }9 |
- if(write(fd, buffer, strlen(buffer)) < 0)" y3 a4 j8 P- @" [
- break;</pre><p style="box-sizing: border-box; border: 0px; font-size: 16px; vertical-align: baseline; line-height: 26px; color: rgb(51, 51, 51); font-family: " helvetica="" neue",="" helvetica,="" tahoma,="" arial,="" "microsoft="" yahei",="" "hiragino="" sans="" gb",="" "wenquanyi="" micro="" hei",="" sans-serif;"=""></p>
复制代码另外需要说明的是,TCP服务器线程在发送MJPEG流的时候是死循环发送的,因此TCP客户端在发送完GET指令之后,就会收到TCP服务器循环发送的图像缓存,TCP客户端会陷入忙等待状态无法再对外发送任何GET或者POST指令,从客户端使用者角度来看的效果就是网页一直在等待。  * H1 ]1 y0 }" j0 S
 / _) ^2 j% p( @
二、UDP上位机UDP发送操作,同样需要先建立UDP Socket:
& I* [3 p- m- O; a/ }& H- int UDP_Send_Found(socklen_t* socket_found , struct sockaddr_in *addr , char* ip , int port)
. Y# T6 \% x: t/ i7 N# R* f - { N# w2 {1 h1 L3 O- }) v: u
- *socket_found = socket(AF_INET, SOCK_DGRAM, 0);( ^% ?0 ^$ r: s% F: c$ @
- if(*socket_found == (~0))
1 h/ w* s h4 b - {: v2 z# w2 Z' p( ?; N$ l
- printf("Create udp send socket failed!\n");9 {$ {: h8 J. |# ]" r
- return -1;/ G# z) Z. y" A, a- o1 \
- }
' t8 a) k; j' z( L* |* B - addr->sin_family = AF_INET;
* ]% ~8 n$ D0 h - addr->sin_addr.s_addr = inet_addr(ip);
: X4 {$ C. f9 O) P9 M! u# Z - addr->sin_port = htons(port);. K Y+ K: @- w! U' O u
- memset(addr->sin_zero, 0, 8);
/ L ]4 n3 c+ f2 L0 n9 e v - return 0;9 s% e% j% T0 ] W$ b$ [8 k/ X
- }
复制代码
* ^0 e; ?8 p5 S& g; y% M# u' X+ ]$ R+ x" j {: B
而UDP文件发送则要比HTTP发送简单得多,只需要将文件切片,每一片为固定长度的UDP帧长度,逐帧发送即可:; q3 i, U0 ~4 A, z$ g3 r
U ~, s+ P8 a" E) V0 V% s3 Z; e& y; p; E9 D
- while(fend > 0)
0 [- |) Y; \ d% l( _# B - {
4 L4 P! Z9 r+ o$ a4 j - memset(picture.data , 0 , sizeof(picture.data));! e# [" c* k) K, p: l2 e- _
- fread(picture.data , UDP_FRAME_LEN , 1, fp);
/ b$ `( `8 r5 Q$ X" P - if(fend >= UDP_FRAME_LEN)
$ f0 H. ?9 M, |/ f - {
& c) f A3 h6 G' q, R - picture.length = UDP_FRAME_LEN;) m' X* \, H& b0 y Z0 p
- picture.fin = 0; ]( `& {5 H1 t
- }
# Z5 Q: J5 a1 w; o5 M1 O8 O; E3 o - else
( _) _+ ^3 J* c& ~, y- n - {( E7 f6 J( S: B3 X6 M) H
- picture.length = fend;, ~3 N! t7 z# e/ }& D1 S
- picture.fin = 1;
' N% M/ e0 h. _ P/ w - }6 R2 {& }% A2 E: I; ?) j
- //printf("sendbytes = %d \n",sendbytes);! c: F. {9 j0 Y$ G4 g2 h
- sendbytes = sendto(socket_send, (char *)&picture, sizeof(struct Package), 0, (struct sockaddr*)&addr,addr_len);
, l& H* \2 Z8 ^% Y - if(sendbytes == -1)
/ m3 m' ?) ?- h" J - {
6 e( [3 ~3 x$ i" f& c - printf("Send Picture Failed!d\n");; p. |9 o3 d7 R' k+ V
- return -1;- T2 I. h8 d$ G2 b9 P0 c2 i: C7 F7 A
- }
) M9 I/ l, ?& n8 r; `: }. @% s, g - else' U5 y! q; W8 t: n( R z% H9 ^
- {
; [& \4 Y9 t: V6 z( v8 h# P - fend -= UDP_FRAME_LEN;
! Z: [8 K$ o3 R: j5 w& _ - }2 ]- z' p( W0 B- y5 D7 o
- }
复制代码
! _5 Y/ e' f* n8 X: A% z! k: D/ c/ {0 W/ g- j8 h0 c4 D% E& ?

: n' X/ G9 k, }9 E+ g. |
! j4 x; Z; a+ c$ X! P; PiMX8MPlus 核心板: https://www.forlinx.com/product/136.html |