本帖最后由 飞凌-marketing 于 2022-1-27 11:00 编辑 ) g4 C' B4 T' w4 [
9 e& {& T6 S0 t3 a

6 ^2 F0 E3 n% W8 W作者|donatello1996 来源 | 电子发烧友 题图|飞凌嵌入式 iMX8MPlus 核心板: https://www.forlinx.com/product/136.html
: p% S9 D: v8 \, @; K4 w& c& E% I; o5 E. R* q
本文采用的硬件板卡为飞凌嵌入式OKMX8MP-C开发板,系统版本Linux5.4.70+Qt5.15.0,主要介绍基于HTTP网页服务器和UDP上位机的MJPG码流传输。 MJPG格式作为一种持续传输的视频码流,在远程监控领域中应用较广,而实现这种远程监控的第三方应用最常见的有两种:浏览器HTTP网页、UDP上位机。 % C, P9 H# l2 }% P/ b S2 o9 C* A
 . z3 P% ]! d7 U1 n3 g1 Z
两者各有优势,对比鲜明,其中: 这两种应用各有优缺点,对于嵌入式开发者来说,两者都必须掌握。
: N2 z& D' b& P% G一、HTTP网页服务器 @9 z3 u _: [& }
先说下HTTP网页服务器获取MJPG码流的代码,首先是OKMX8MP-C在开发板端建立TCP服务器: - int TCP_Server_Found(socklen_t* socket_found , char* ip , int port)! p2 R7 ], g( B$ L8 Q- S. E2 N0 g
- {
7 [- M0 }- U$ R$ E4 y7 m, n0 E- A3 X# ^ - struct sockaddr_in servaddr;4 k" }3 ] O" t6 ?2 q
- socklen_t addrsize = sizeof(struct sockaddr);
& e/ H' c. K! m, d$ C! [& b - bzero(&servaddr , sizeof(servaddr));' _" K# m$ s; c7 u) N
- servaddr.sin_family = AF_INET;
' j# v( A; I. u, W8 E* \) N+ q0 ~, _ - servaddr.sin_addr.s_addr = inet_addr(ip);
. t; t6 f6 u( n( ^ - servaddr.sin_port = htons(port);
8 ^2 b# C4 O' e) Q( P( ~1 z - int ret;
' S8 i. J- V5 s# C - IF( (*socket_found = socket(AF_INET , SOCK_STREAM , 0)) == -1)' \% [8 \! q. v& W5 a+ M
- {9 G; v6 a" |' R7 `$ j% e
- printf("Create socket error: %s (errno :%d)\n",strerror(errno),errno); G9 l% T t [7 P Z( b
- return -1;
( n* v5 y) J% s5 A! p - }% g* h" \$ X" _: u+ z4 e
- int on = 1;
) F6 A* y( K7 A% l9 c0 c - if(setsockopt(*socket_found , SOL_SOCKET, SO_REUSEADDR, &on, sizeof(on)) < 0)
3 _3 `5 r' B& I+ P. {' C - {0 g0 g! }$ v0 N2 K) ?, J% l# W
- printf("setsockopt error\n");- G/ Z* z3 s8 b: `* h3 I! }$ w
- }
/ q9 z7 k; F Q - ret = bind(*socket_found , (struct sockaddr *)&servaddr , addrsize);, I* M1 P1 A# p/ @( Y. ]) ^& K" K
- if(ret == -1)
) s4 e, \5 t x5 B4 F - {
: w" ?6 D9 l2 { - printf("Tcp bind faiLED!\n");
1 i( q- c, {2 j9 n5 i! a - return -1;
& ^/ n9 x* `% i$ Y) }! Q9 B& N - }
. T+ x! q0 t& a. b - if(listen(*socket_found , 5) == -1)3 d, Y. s( X# e7 i( b
- {+ s( l- U+ W. d* \2 w
- printf("Listen failed!\n");( h6 x6 c8 i$ ?
- return -1;
2 h9 S& O$ M5 e. x2 B; d - }: ~9 r2 S. O/ y O& K E
- return 0;& A$ [7 r; f! }0 x2 w4 R# [
- }
复制代码其中setsockopt()函数是可选的,一般只用于规避socket()函数的建立错误。 建立了TCP服务器后,返回的socklen_t型实参在后面的HTTP网页服务器中需要用到。 HTTP网页服务器所属的TCP操作是需要另起轮询线程来让客户端进行accept()握手操作的,accept()之前的listen()倒是只需要执行一次即可,accept()握手操作和recv()接收操作需要创建一个死循环线程: - pthread_create(&tid_tcp_web_recv , NULL , Thread_TCP_Web_Recv , NULL);
6 N4 E1 i+ \& C( U - void * Thread_TCP_Web_Recv(void *arg)
- t# e, {. T" y' r - {
9 d/ w- R" e& ~4 D( ~; P' j$ v0 }" u - 。。。
# P' x) v4 ?3 f - while(1), d. g8 k2 y3 V
- {
9 ]9 f+ K7 G c2 [( ~5 D$ C - fd_socket_conn = accept(socket_web_server , (struct sockaddr *)&sockaddr_in_conn , &addrsize);* ]; r& d& U$ v; E
- printf("fd_socket_conn = accept()\n");
" b! k( k5 f& j: Q - 。。。3 e \. G- r% Q/ v5 T' T6 n
- recv(fd_socket_conn , recvbuf , 1000 , 0);, k3 `) ]- W6 r
- }8 T- N3 C" g! Y; K8 l
- 。。。! M& } c" H. a3 V+ T9 K3 C
- }
复制代码MJPG帧可以使用Grab操作获取,获取到的MJPG帧需要在TCP线程中读,在Grab操作线程中写,这种被多个线程访问的资源需要加锁防止读写冲突,即资源被Grab操作写入时,需要上锁,不允许其它线程访问,操作完成时需要解锁,允许其它线程访问: - pthread_mutex_lock(&pmt);7 E1 T, l' Z$ B8 J, m" U% s+ x- b6 b$ i
- pic_tmpbuffer = pic.tmpbuffer;5 }/ ]5 E# \/ \4 m {
- pic.tmpbytesused = buff.bytesused;2 A( S3 }7 b7 n' V5 P; _
- pic_tmpbytesused = pic.tmpbytesused; U1 b( f- X& T! g/ t0 h
- pthread_cond_broadcast(&pct);1 R' i* } A7 z2 A3 F+ J
- pthread_mutex_unlock(&pmt);
复制代码线程互斥锁使用之前需要初始化: - pthread_mutex_t pmt;
& a* U2 w5 v ]- i7 U( A* { - pthread_cond_t pct;
- D" P% {# v; {5 H0 G3 J! y9 {, }: ] - int main(int argc, char* argv[])
& m. I+ V& A) Z+ W6 Q5 R z: j - {
3 U4 z5 Y, ~& m3 f% D4 V; v - ..." [' c! I( R. }6 v$ Y3 f7 k
- TCP_Server_Found(&socket_web_server , (char*)argv[2] , PORT_TCP);
6 ~3 e, a7 ?5 D% w% ~ - pthread_mutex_init(&pmt , NULL);6 j. l& t$ W4 Q
- pthread_create(&tid_tcp_web_recv , NULL , Thread_TCP_Web_Recv , NULL);
" G4 P/ Y" c; V i - pthread_create(&tid_tcp_web_send , NULL , Thread_TCP_Web_Send , NULL);
+ U3 O3 U6 k" S% l0 \/ E8 i - ...
* [4 [) q; Q1 X1 j - while(1)
6 H3 W) W. R2 {; Y, _ - {
" s/ E( o( a, `" j - V4l2_Grab_Mjpeg(false , MJPEG_FILE_NAME);
$ X1 X# ?5 K* B$ B* e6 y - ...
0 z$ ^3 Y+ a9 `6 | - }
5 d# r2 P$ }/ P k5 G4 E - ...5 Q0 X6 |+ G+ M; J, Y
- }
复制代码然后是发送的细节,发送图片文件之前,需要先发送HTTP标准头,这个相当于给发送图片或者其它类型的流数据铺路: - <p style="box-sizing: border-box; border: 0px; vertical-align: baseline; line-height: 26px;"><span style="text-indent: 32px;">
, r7 h- q6 t* d2 `0 A; h3 \: P4 n - </span></p><p style="box-sizing: border-box; border: 0px; vertical-align: baseline; line-height: 26px;"></p>
复制代码- #define STD_HEADER "Connection: close\r\n" \4 U, O( F7 H% g8 {* h" o
- "Server: MJPG-Streamer/0.2\r\n" \. O* L" G: L6 |: c( }) ^3 J
- "Cache-Control: no-store, no-cache, must-revalidate, pre-check=0, post-check=0, max-age=0\r\n" \. F7 z) r% q ?6 i* P
- "Pragma: no-cache\r\n" \
+ W- p$ b* F# I0 y( ~- i - "Expires: Mon, 3 Jan 2000 12:34:56 GMT\r\n"
' y% Z' \8 \7 K# M# h - #define BOUNDARY "boundarydonotcross"
* G/ R7 L) x! D: Q9 p - printf("preparing header\n");
; X3 T7 V8 V5 h2 K P1 s" d( | - sprintf(buffer, "HTTP/1.0 200 OK\r\n" \
! b* d4 Z, l6 d; O, k - "Access-Control-Allow-Origin: *\r\n" \
) m( Q: b% b8 W, y - STD_HEADER \
1 ~/ |" q7 E% [ i, w+ G2 t7 k { - "Content-Type: multipart/x-mixed-replace;boundary=" BOUNDARY "\r\n" \* |9 o" ^5 g% x* ^% `
- "\r\n" \6 N( R9 a% d+ W2 e4 L+ V+ D
- "--" BOUNDARY "\r\n");
6 L7 ~7 `8 H, g5 h. m3 S: C* F0 d - if(write(fd, buffer, strlen(buffer)) < 0)
5 m) ]- O% ]1 p - {! c0 w9 b: C0 b
- free(frame);) ^! Q5 c5 |9 K. y3 d4 U
- return;+ c4 H' v% e5 q5 p: x' L% c3 v
- }
复制代码发送完HTTP标准头之后,就需要发送内容头(Content-Type),这处的Content-Type为image/jpeg,同样,HTTP标准协议里面image支持的类型远不止jpeg一种,发送完内容头之后就是正文和boundary结尾,这样帧完整的HTTP头发送到指定的TCP GET地址,就会在浏览器中显示刚刚发送的图片: - <pre class="prettyprint lang-cpp" style="box-sizing: border-box; font-family: Monaco, Menlo, Consolas, "Courier New", monospace; font-size: 16px; white-space: pre-wrap; line-height: 1.38462; color: rgb(51, 51, 51); word-break: break-all; background-color: rgb(245, 245, 245); border: 0px; border-radius: 4px; vertical-align: baseline;"> sprintf(buffer, "Content-Type: image/jpeg\r\n" \$ N8 D6 R, l( h, Y* X; r; n6 a
- "Content-Length: %d\r\n" \4 D" |8 ?" Q. ?
- "X-Timestamp: %d.%06d\r\n" \2 W. f7 _" r! G1 r
- "\r\n", frame_size, (int)timestamp.tv_sec, (int)timestamp.tv_usec);/ | n0 ?4 D( Y1 P" B+ R6 m
- printf("sending intemdiate header\n");! L+ E" H' Q/ S* c1 x7 A" {
- if(write(fd, buffer, strlen(buffer)) < 0)( T: U2 t* q9 l' S
- break;$ Y; n4 Q6 {1 w/ h$ I
- printf("sending frame\n");
7 U( T U" v. Z7 C3 D) _* a- X( R4 Q - if(write(fd, frame, frame_size) < 0)# }* i( O F5 f/ ]- W
- break;
. M# W) c+ e1 Q$ v' \6 r8 o) `" m: W - printf("sending boundary\n");# z: J8 F: h$ z1 ?
- sprintf(buffer, "\r\n--" BOUNDARY "\r\n");; I' [6 y) f3 _4 @. L
- if(write(fd, buffer, strlen(buffer)) < 0)
+ n' y. b; p, c( o - break;</pre><p style="box-sizing: border-box; border: 0px; font-size: 16px; vertical-align: baseline; line-height: 26px; color: rgb(51, 51, 51); font-family: " helvetica="" neue",="" helvetica,="" tahoma,="" arial,="" "microsoft="" yahei",="" "hiragino="" sans="" gb",="" "wenquanyi="" micro="" hei",="" sans-serif;"=""></p>
复制代码另外需要说明的是,TCP服务器线程在发送MJPEG流的时候是死循环发送的,因此TCP客户端在发送完GET指令之后,就会收到TCP服务器循环发送的图像缓存,TCP客户端会陷入忙等待状态无法再对外发送任何GET或者POST指令,从客户端使用者角度来看的效果就是网页一直在等待。  ) Y2 ]% K v* y k3 [# s4 V
 6 ]* v4 Z1 Z9 c
二、UDP上位机UDP发送操作,同样需要先建立UDP Socket:
; `' `# S0 j S4 V2 k- int UDP_Send_Found(socklen_t* socket_found , struct sockaddr_in *addr , char* ip , int port)! V5 s M* }8 S# O- u" V6 ?! U
- {/ T2 C9 l8 L. G
- *socket_found = socket(AF_INET, SOCK_DGRAM, 0);
1 I; b4 u; T; E9 ` - if(*socket_found == (~0))4 _; H+ g2 O& R" R& _* \
- {, d" i# d( f+ B+ x& `4 K
- printf("Create udp send socket failed!\n");, n5 C7 b/ ?- D A
- return -1;3 u* m7 h+ W% T/ G% F6 c. s
- }
' u. P4 ~2 z; K4 D f0 p8 o - addr->sin_family = AF_INET;
6 T. D: [: R Q2 x/ S - addr->sin_addr.s_addr = inet_addr(ip);! Y; Y* _4 D M" l$ }- b3 ^# i
- addr->sin_port = htons(port);8 b. x3 s% Q/ a, E7 b- O: v" Y$ @
- memset(addr->sin_zero, 0, 8);# Z3 s5 E" ?( a1 s: Q; V- P t2 o
- return 0;
1 J0 v8 m0 `, O- d `+ L - }
复制代码 ( @7 P% Z1 h. C& B. l5 M
$ t+ Z; c$ }: A" i! e7 m5 _: _% v' S
而UDP文件发送则要比HTTP发送简单得多,只需要将文件切片,每一片为固定长度的UDP帧长度,逐帧发送即可:
6 u d2 {4 f2 Q ]+ c. l/ ]/ O; L+ E0 M' e' n0 G$ z
8 X0 \2 E, S( i" c' C9 Z) h- while(fend > 0)
! x3 s- q- o e' K - {
, s" d0 w: @: c `8 U: |4 h4 c - memset(picture.data , 0 , sizeof(picture.data));
% `! v$ l( y) p; \6 P - fread(picture.data , UDP_FRAME_LEN , 1, fp);9 c4 T) Y. N! P; |/ _& ~! F
- if(fend >= UDP_FRAME_LEN). B. G$ G6 k# a+ Z( p
- {$ i: |2 M1 ~$ t+ X
- picture.length = UDP_FRAME_LEN;
* @3 {- f! k) w$ A - picture.fin = 0;
- A, p. n q4 v, R/ B- u/ o - }
/ i3 C4 j+ n8 L7 ] - else, p2 S8 F$ F* Y" L
- {% ~, X2 Y0 S1 [# ^8 o: B
- picture.length = fend;
0 f- H" M/ T( c - picture.fin = 1;4 }: n" ?* {, k; h( a9 B
- }, m2 A7 p: G& O- s- i+ _, Q# I
- //printf("sendbytes = %d \n",sendbytes);
2 r, P2 @$ R6 l, d8 w k/ V; { - sendbytes = sendto(socket_send, (char *)&picture, sizeof(struct Package), 0, (struct sockaddr*)&addr,addr_len);0 m# ?# K8 f/ b% w" e
- if(sendbytes == -1)( D, T+ B7 H6 n3 @# g
- {
% v% X) H' W- b' o+ R7 n: z4 ^ - printf("Send Picture Failed!d\n");% D) b3 s* s9 S+ V7 q$ h( V7 v
- return -1;
% ?- b5 O l8 ?/ B2 ] q - }/ Z* I/ F9 N( s6 g
- else
, _& O6 Y9 |$ w$ q/ @* J: V% e2 ` - {
4 w: Q* a$ i$ l3 s, z0 E - fend -= UDP_FRAME_LEN;
( L; j R& l9 j - }
5 B& ?% U( m$ r0 j+ f3 |" f - }
复制代码
0 s7 o; T* H( M# e6 v3 I7 x6 i7 E+ p2 `
 3 C/ w6 ]% I! X* f$ |% i9 o1 H
7 E$ m( V' e( q: o" Z
iMX8MPlus 核心板: https://www.forlinx.com/product/136.html |