本帖最后由 飞凌-marketing 于 2022-1-27 11:00 编辑
7 K* G. c7 t+ q7 N2 R. Q1 c! A4 y- p( A3 X4 j/ J" w: c* G* o

* O( w& S( j: E! m作者|donatello1996 来源 | 电子发烧友 题图|飞凌嵌入式 iMX8MPlus 核心板: https://www.forlinx.com/product/136.html7 w5 o% e- H6 i* G4 n
- b& C7 N! J* ~, B. D
本文采用的硬件板卡为飞凌嵌入式OKMX8MP-C开发板,系统版本Linux5.4.70+Qt5.15.0,主要介绍基于HTTP网页服务器和UDP上位机的MJPG码流传输。 MJPG格式作为一种持续传输的视频码流,在远程监控领域中应用较广,而实现这种远程监控的第三方应用最常见的有两种:浏览器HTTP网页、UDP上位机。
. e% G4 f5 }: M+ ^
# i0 m3 i9 q4 l* S, g$ q- L两者各有优势,对比鲜明,其中: 这两种应用各有优缺点,对于嵌入式开发者来说,两者都必须掌握。 ( [0 N$ \* `6 ~8 g8 I2 N- k2 B2 E) l
一、HTTP网页服务器
& d$ y" p* l* [+ T' N7 c; F先说下HTTP网页服务器获取MJPG码流的代码,首先是OKMX8MP-C在开发板端建立TCP服务器: - int TCP_Server_Found(socklen_t* socket_found , char* ip , int port)8 B+ I7 ]8 j( o c' \, v
- {
5 u( d& u6 N( a! P0 g6 s) p - struct sockaddr_in servaddr;
! t2 r9 o1 g5 H" s) W% P* w- D2 ? - socklen_t addrsize = sizeof(struct sockaddr);/ e/ f: d& t" y6 j
- bzero(&servaddr , sizeof(servaddr));, A, k( Z; a9 Q, `) \0 s* m
- servaddr.sin_family = AF_INET;2 g1 @% b! B( m* y% A* f: z
- servaddr.sin_addr.s_addr = inet_addr(ip);
- w! W" g/ ^- m# L* b0 b' F R - servaddr.sin_port = htons(port);
- t: I& q# S/ g7 g m0 g - int ret;
1 O0 k" j k6 A y( e6 M - IF( (*socket_found = socket(AF_INET , SOCK_STREAM , 0)) == -1)$ C+ m/ d$ U/ u5 N) t4 T
- {+ b" q" M/ [, N" T
- printf("Create socket error: %s (errno :%d)\n",strerror(errno),errno);
3 F5 p) O3 l$ k m$ Q! X - return -1;
( A, o. P# x, ~ - }; _/ |8 X6 t7 e# c# ^: [
- int on = 1;
' F- ~" O/ {9 V' f# I5 g3 h - if(setsockopt(*socket_found , SOL_SOCKET, SO_REUSEADDR, &on, sizeof(on)) < 0)% y3 R/ @4 [" \; V8 I
- {
3 ~9 z! `! F$ S$ N- D+ b3 y - printf("setsockopt error\n");
* Y H2 k' _4 q7 A0 a q - }
- P) t0 O Q3 h/ L1 e; o - ret = bind(*socket_found , (struct sockaddr *)&servaddr , addrsize);
6 e# e$ \) L A& \4 E6 F - if(ret == -1)
% i7 h) ]2 c* ^: ]$ j - {
9 v- s, l* Z* B/ f/ ~" P% o1 d - printf("Tcp bind faiLED!\n");
) G! ]) A/ O" `% C# b- u - return -1;
1 v8 m6 X3 Y5 \: y" m& Y$ g - }
7 a }/ c' C' s' _+ W* E2 o, x - if(listen(*socket_found , 5) == -1)
- ~5 Y& q6 ?* [' y( j' [ - {4 A$ M4 g2 e; n& u+ A8 K5 l' p) y
- printf("Listen failed!\n");! [0 l: x# E2 i8 D% m
- return -1;$ x3 I% V3 r6 s. n
- }2 p9 _1 h5 P# q$ L. a$ o
- return 0;
5 R$ t$ b9 r! B% g& ]; x - }
复制代码其中setsockopt()函数是可选的,一般只用于规避socket()函数的建立错误。 建立了TCP服务器后,返回的socklen_t型实参在后面的HTTP网页服务器中需要用到。 HTTP网页服务器所属的TCP操作是需要另起轮询线程来让客户端进行accept()握手操作的,accept()之前的listen()倒是只需要执行一次即可,accept()握手操作和recv()接收操作需要创建一个死循环线程: - pthread_create(&tid_tcp_web_recv , NULL , Thread_TCP_Web_Recv , NULL);
4 i% m2 K4 R# y+ T7 F/ e' ^- g+ Q - void * Thread_TCP_Web_Recv(void *arg)' Z3 V' ]3 c }; G8 H, e
- {% Q7 ~$ l! o9 Y( K5 C$ G2 X
- 。。。
$ E& s' P! X2 @- d& d) n - while(1)
, j: B7 t0 D7 \0 @ - {# s% U! N/ G# W7 Y9 [$ S- d+ I2 l
- fd_socket_conn = accept(socket_web_server , (struct sockaddr *)&sockaddr_in_conn , &addrsize);9 C' O6 K+ Z. E0 S) w/ g
- printf("fd_socket_conn = accept()\n");( ^1 [2 b' ~) V0 e$ U, [" `9 m
- 。。。
: Z$ p. g* x6 [: g - recv(fd_socket_conn , recvbuf , 1000 , 0);
- N8 z: g9 V6 B. l2 @2 ?) D. _. M, s - }, a; U: o$ U1 Q, v
- 。。。8 A+ C& z/ @( z6 t$ [' K0 h, m2 D
- }
复制代码MJPG帧可以使用Grab操作获取,获取到的MJPG帧需要在TCP线程中读,在Grab操作线程中写,这种被多个线程访问的资源需要加锁防止读写冲突,即资源被Grab操作写入时,需要上锁,不允许其它线程访问,操作完成时需要解锁,允许其它线程访问: - pthread_mutex_lock(&pmt);! T% Y% X7 }/ W, E1 a% V1 [
- pic_tmpbuffer = pic.tmpbuffer;
1 J2 a/ |8 V" }7 v) V: } - pic.tmpbytesused = buff.bytesused;
+ u3 E0 C) b$ g1 A$ P N0 k5 J - pic_tmpbytesused = pic.tmpbytesused;
o% P. ~7 S' r+ N& } ~( c - pthread_cond_broadcast(&pct);
4 H, E% {! K6 w$ M% \ - pthread_mutex_unlock(&pmt);
复制代码线程互斥锁使用之前需要初始化: - pthread_mutex_t pmt;
W4 X7 Q l# C3 A6 {( z/ [ - pthread_cond_t pct;+ M' d4 |% L- C y4 W5 C* j
- int main(int argc, char* argv[])# ~+ P: o9 z" w7 [
- {3 S# W0 h& O9 H: v. N
- ...
" Y' R; V( u$ e! M1 Z - TCP_Server_Found(&socket_web_server , (char*)argv[2] , PORT_TCP);! G7 Z) u, h- C' `2 k7 |8 p3 b
- pthread_mutex_init(&pmt , NULL);+ I0 Y/ s3 L7 I5 M" O
- pthread_create(&tid_tcp_web_recv , NULL , Thread_TCP_Web_Recv , NULL);
' t4 E2 w" |4 P5 P! m) X! t* | - pthread_create(&tid_tcp_web_send , NULL , Thread_TCP_Web_Send , NULL);
# Y" l* u6 }( |1 Z' Y1 P - ...
6 u( N' c- W, d- E6 A - while(1): Y7 R; N" i9 |6 f$ Z
- {
, o$ M+ O6 _; y5 p. x, `3 z, [ - V4l2_Grab_Mjpeg(false , MJPEG_FILE_NAME);
# n: r/ ?9 ~, B, q' t0 A" [; M P - ...
! y! z0 E' U5 r }! X& ^, S - }6 R- @3 E* V3 d7 N2 \
- ...8 k/ l$ c' |' ]! y! F! F
- }
复制代码然后是发送的细节,发送图片文件之前,需要先发送HTTP标准头,这个相当于给发送图片或者其它类型的流数据铺路: - <p style="box-sizing: border-box; border: 0px; vertical-align: baseline; line-height: 26px;"><span style="text-indent: 32px;">6 ]0 I9 r& ?/ |1 T- }# w$ \1 D: E4 R
- </span></p><p style="box-sizing: border-box; border: 0px; vertical-align: baseline; line-height: 26px;"></p>
复制代码- #define STD_HEADER "Connection: close\r\n" \
+ M* a6 j/ A) A' } - "Server: MJPG-Streamer/0.2\r\n" \
" o/ o# c: x: A( k - "Cache-Control: no-store, no-cache, must-revalidate, pre-check=0, post-check=0, max-age=0\r\n" \ M2 W: f" D' i: n; I" B( `% v
- "Pragma: no-cache\r\n" \) x+ f% x+ G6 u7 R" }. p
- "Expires: Mon, 3 Jan 2000 12:34:56 GMT\r\n"
- S* T- N: i. i7 b/ Z3 Q - #define BOUNDARY "boundarydonotcross"
0 I% Q1 K3 v8 C5 l/ Z0 M) f - printf("preparing header\n");$ v7 ~. M& b2 `4 H1 d! q
- sprintf(buffer, "HTTP/1.0 200 OK\r\n" \6 v: c3 m9 ?2 F7 u# u- Z h5 \# O
- "Access-Control-Allow-Origin: *\r\n" \6 @# w% e$ B. l% Y
- STD_HEADER \3 C* _% N8 n: b. H4 |9 J X) [1 z
- "Content-Type: multipart/x-mixed-replace;boundary=" BOUNDARY "\r\n" \
5 S, ^0 u! g6 O+ A+ u, v - "\r\n" \
9 P4 n% I0 H8 Y' ?, d0 T - "--" BOUNDARY "\r\n");
9 `/ v. ^0 N- \; X0 o- c9 @ - if(write(fd, buffer, strlen(buffer)) < 0)
) V4 Y1 [4 h+ x4 ?$ H) E9 M% R - {
/ K( ?( `5 q# m5 n) J - free(frame);1 Y* ~' y8 t" ^8 p0 x) J
- return;
" ]4 \2 a' E' g8 ]1 W( b; a - }
复制代码发送完HTTP标准头之后,就需要发送内容头(Content-Type),这处的Content-Type为image/jpeg,同样,HTTP标准协议里面image支持的类型远不止jpeg一种,发送完内容头之后就是正文和boundary结尾,这样帧完整的HTTP头发送到指定的TCP GET地址,就会在浏览器中显示刚刚发送的图片: - <pre class="prettyprint lang-cpp" style="box-sizing: border-box; font-family: Monaco, Menlo, Consolas, "Courier New", monospace; font-size: 16px; white-space: pre-wrap; line-height: 1.38462; color: rgb(51, 51, 51); word-break: break-all; background-color: rgb(245, 245, 245); border: 0px; border-radius: 4px; vertical-align: baseline;"> sprintf(buffer, "Content-Type: image/jpeg\r\n" \
- j0 u! ^' i y D& z0 E' H - "Content-Length: %d\r\n" \! g" Z6 z3 }/ I% l+ }) r
- "X-Timestamp: %d.%06d\r\n" \
: f. f% _! ` E0 a T9 n - "\r\n", frame_size, (int)timestamp.tv_sec, (int)timestamp.tv_usec);; L/ a2 ~" T4 C/ L* ~
- printf("sending intemdiate header\n");6 L( \. f$ h4 R y, G4 U
- if(write(fd, buffer, strlen(buffer)) < 0)
- E( ?0 a3 o- o8 {+ f ^ - break;
& I; L8 J' m" o$ g0 a6 J$ i: K - printf("sending frame\n");" o B |4 c- }2 F4 K
- if(write(fd, frame, frame_size) < 0)% s; d# P' g$ u6 O! A) G" c
- break;* y2 }; U+ `# l
- printf("sending boundary\n");
2 c9 w; B; n! X. b' N8 d - sprintf(buffer, "\r\n--" BOUNDARY "\r\n");) ~0 F8 {9 t3 J! ~5 z, \
- if(write(fd, buffer, strlen(buffer)) < 0)4 \3 ^4 Q1 o. h6 f
- break;</pre><p style="box-sizing: border-box; border: 0px; font-size: 16px; vertical-align: baseline; line-height: 26px; color: rgb(51, 51, 51); font-family: " helvetica="" neue",="" helvetica,="" tahoma,="" arial,="" "microsoft="" yahei",="" "hiragino="" sans="" gb",="" "wenquanyi="" micro="" hei",="" sans-serif;"=""></p>
复制代码另外需要说明的是,TCP服务器线程在发送MJPEG流的时候是死循环发送的,因此TCP客户端在发送完GET指令之后,就会收到TCP服务器循环发送的图像缓存,TCP客户端会陷入忙等待状态无法再对外发送任何GET或者POST指令,从客户端使用者角度来看的效果就是网页一直在等待。  + a6 z- @! q9 ~' K, i' H% h1 d

T; c4 X" @; O2 g+ l$ i( ~二、UDP上位机UDP发送操作,同样需要先建立UDP Socket:
5 L# t- _& G) I1 ?$ g6 y% | y- int UDP_Send_Found(socklen_t* socket_found , struct sockaddr_in *addr , char* ip , int port)
% b* [+ M7 _6 q( N$ y- k - {3 f5 [, X+ H) z
- *socket_found = socket(AF_INET, SOCK_DGRAM, 0);
2 D9 V: ]- ^5 P8 f' |, e - if(*socket_found == (~0)): M' n. m0 P. V" V2 H* a: ?0 O! s
- {
; b4 W9 y! y1 Q9 n3 y( }6 R - printf("Create udp send socket failed!\n");! {) k" L3 j2 O1 Z* K
- return -1;$ a, m& T- v( u! r3 p( H$ I* {
- }1 X& x4 s7 Y' ?( r: p7 A- v
- addr->sin_family = AF_INET;# e4 v& m6 B) B9 l1 z6 S! D2 {% N4 H
- addr->sin_addr.s_addr = inet_addr(ip);
& ?6 y% C" B7 g8 j) t - addr->sin_port = htons(port);
* K9 V7 I! x F: `* W - memset(addr->sin_zero, 0, 8);1 m! K; W& Z% f4 m) ?
- return 0;: t, v) z. w# k2 |
- }
复制代码
+ A8 h" w" s0 j4 w# P+ f, Z# Z$ D! U# B( W# I
而UDP文件发送则要比HTTP发送简单得多,只需要将文件切片,每一片为固定长度的UDP帧长度,逐帧发送即可:1 J4 E, N! W- ~% s: Y
$ V2 d2 a! v- s* d/ G
) @% r6 p6 s' ]: G6 I$ {- while(fend > 0)8 P6 r: I5 S L+ i* Y
- { R$ N ^$ R, |; g9 K( ]* e
- memset(picture.data , 0 , sizeof(picture.data));
' G( o9 V' T2 D - fread(picture.data , UDP_FRAME_LEN , 1, fp);
) |3 A* |+ d( t: d2 D4 Q. B- l7 b- r - if(fend >= UDP_FRAME_LEN)
. X n0 \: o3 A4 N% l+ E+ Q5 ? - {
! N- Q2 A: W1 |" l# c' a - picture.length = UDP_FRAME_LEN;
) u. o7 U: ?( z: d: I- O, q3 d - picture.fin = 0;* `8 x2 ?, m" g
- }: V6 f3 m$ i4 j! c0 h, o. x
- else- \/ {3 f5 c) j: c# p' ^& \/ X
- {% s* p% n$ _, u: I" q* N
- picture.length = fend;' K. |2 W' W$ E* i6 T
- picture.fin = 1;* W4 q i: t; q; m
- }- N* {) H+ M9 g7 @5 _
- //printf("sendbytes = %d \n",sendbytes);
$ Y. F# y, m1 Z5 l4 ^ - sendbytes = sendto(socket_send, (char *)&picture, sizeof(struct Package), 0, (struct sockaddr*)&addr,addr_len);! D4 V# ~+ \8 {- p; ?5 q3 z4 L
- if(sendbytes == -1)8 n9 x9 ]* ~. F7 i4 i
- {
4 u4 b' K: w$ N - printf("Send Picture Failed!d\n");2 T# _, `# B7 w2 N3 n* f6 g% v
- return -1;
- O g, \6 j6 U& T) j4 r0 M - }' }' X) h& i. R
- else- I. }5 Z$ C t6 I% F: W
- {0 L! c% K) I6 T1 D }2 z5 f5 _
- fend -= UDP_FRAME_LEN;
& x% t; a- c, s) s - }" A% d9 I6 c, ]" S
- }
复制代码 9 T/ y/ Y1 Q7 ] N! }
! x6 n0 k; c5 a5 k7 P' w! s0 p + p# G" ]; q! A3 C" S* @; J
$ s. |* K$ h. O) U+ V" e! q, wiMX8MPlus 核心板: https://www.forlinx.com/product/136.html |