本帖最后由 飞凌-marketing 于 2022-1-27 11:00 编辑 6 U7 } i' O# }- b* O; p0 i1 l3 G
. N6 F" U5 W' O Z
1 `: Q; S; B7 N$ T5 F+ j5 Y( @; ?作者|donatello1996 来源 | 电子发烧友 题图|飞凌嵌入式 iMX8MPlus 核心板: https://www.forlinx.com/product/136.html
3 r* q3 T7 O* R' T2 W6 x3 l: H5 ^* I. V; y# r) u4 `8 Y8 ^
本文采用的硬件板卡为飞凌嵌入式OKMX8MP-C开发板,系统版本Linux5.4.70+Qt5.15.0,主要介绍基于HTTP网页服务器和UDP上位机的MJPG码流传输。 MJPG格式作为一种持续传输的视频码流,在远程监控领域中应用较广,而实现这种远程监控的第三方应用最常见的有两种:浏览器HTTP网页、UDP上位机。 " _" E) K8 B6 j' f
 * E3 o7 s" m3 L% _3 p# N+ p J
两者各有优势,对比鲜明,其中: 这两种应用各有优缺点,对于嵌入式开发者来说,两者都必须掌握。 Y; b/ \, z# s+ B- y' e% Q) ?0 R
一、HTTP网页服务器4 ~+ }- F; b- A
先说下HTTP网页服务器获取MJPG码流的代码,首先是OKMX8MP-C在开发板端建立TCP服务器: - int TCP_Server_Found(socklen_t* socket_found , char* ip , int port) p6 b _) v# A8 x1 P
- {
# H4 a5 f# Z' ?" d( F! ?$ N H - struct sockaddr_in servaddr;
9 F* o3 J! Q3 n$ I9 B - socklen_t addrsize = sizeof(struct sockaddr);
5 l# S; t+ @: X - bzero(&servaddr , sizeof(servaddr));( A' [7 i1 `8 O1 v; o; \' u
- servaddr.sin_family = AF_INET;
* v$ S X, q Q. V8 T - servaddr.sin_addr.s_addr = inet_addr(ip);! q/ I+ [) m# V/ s3 f# [
- servaddr.sin_port = htons(port);* e" S, A. d7 ]# Z3 @
- int ret;
E, D5 x5 ~# O+ ~1 d2 S - IF( (*socket_found = socket(AF_INET , SOCK_STREAM , 0)) == -1)
3 T" K6 t8 L8 b, d$ r1 ] - {7 b! D* N/ a; A1 u1 `" r4 m3 E
- printf("Create socket error: %s (errno :%d)\n",strerror(errno),errno);
6 e7 B& O, W7 Z8 {1 a) B: u/ ] - return -1;
4 j9 P, h6 ]! w - }
7 s( a# K# e5 r) d/ h% M - int on = 1;4 p2 j0 [. {1 [) [( j. H& F% @
- if(setsockopt(*socket_found , SOL_SOCKET, SO_REUSEADDR, &on, sizeof(on)) < 0)+ G' S: e$ S1 a- _
- {% S2 e9 H3 w1 p9 Q+ A% h3 v/ h
- printf("setsockopt error\n");
& N' X7 D7 S4 H+ }: o. y - }6 C3 b* e, ]. `, K; c5 H
- ret = bind(*socket_found , (struct sockaddr *)&servaddr , addrsize);; Z6 u$ F+ u. W% r( h6 U* Y
- if(ret == -1)% h+ @2 _: F0 |; @. s# L
- {
; k8 k: s6 j5 e: t5 x5 f - printf("Tcp bind faiLED!\n");' ~; ~& k, S/ Y7 Z5 F, F' r
- return -1;3 l+ y- p7 d8 J2 C7 [, z# o) K
- }
4 V! c# M% C6 \8 S) ?2 \9 u; h - if(listen(*socket_found , 5) == -1)
/ B9 y8 D6 e; }4 k - {
2 P1 Z: C4 E6 F* t; a - printf("Listen failed!\n");
+ G: p8 a3 d5 S6 U/ E5 W) j - return -1;
! j; r3 x' J0 |5 w7 r y - }
# P& t: l, }% k I0 D; j4 ] - return 0;6 K' J ^# y. W3 P7 R. ?, j" D% m) W
- }
复制代码其中setsockopt()函数是可选的,一般只用于规避socket()函数的建立错误。 建立了TCP服务器后,返回的socklen_t型实参在后面的HTTP网页服务器中需要用到。 HTTP网页服务器所属的TCP操作是需要另起轮询线程来让客户端进行accept()握手操作的,accept()之前的listen()倒是只需要执行一次即可,accept()握手操作和recv()接收操作需要创建一个死循环线程: - pthread_create(&tid_tcp_web_recv , NULL , Thread_TCP_Web_Recv , NULL);
5 e5 I! a+ M |/ O' t! Y - void * Thread_TCP_Web_Recv(void *arg)3 _& q: L) ^5 g
- {
( u. P( q' ?6 `# B- p6 ~8 o# N - 。。。- ^/ h" g( I: h o
- while(1)! |: [& R2 K% V8 K& O
- {5 _' Q8 Q" E/ X& e
- fd_socket_conn = accept(socket_web_server , (struct sockaddr *)&sockaddr_in_conn , &addrsize);
3 ]1 K+ b6 j: P - printf("fd_socket_conn = accept()\n");
0 S- J( g% R9 a `3 ` - 。。。
' b3 v" V8 ?1 S9 N7 ?: L - recv(fd_socket_conn , recvbuf , 1000 , 0);
; p2 ], n1 `% m4 G' h - }
]* b/ G* C3 G6 ?) U* U1 r/ { - 。。。
. \3 ^% x% C7 D3 D; T: U - }
复制代码MJPG帧可以使用Grab操作获取,获取到的MJPG帧需要在TCP线程中读,在Grab操作线程中写,这种被多个线程访问的资源需要加锁防止读写冲突,即资源被Grab操作写入时,需要上锁,不允许其它线程访问,操作完成时需要解锁,允许其它线程访问: - pthread_mutex_lock(&pmt);
# c$ f9 ~ A3 m- G; B% r' p4 F - pic_tmpbuffer = pic.tmpbuffer;
6 e6 ?: C: z6 W% `: H - pic.tmpbytesused = buff.bytesused; v! X+ r$ K! K" Q0 B
- pic_tmpbytesused = pic.tmpbytesused;
) X9 w1 ?7 M; K5 g - pthread_cond_broadcast(&pct);! }' N/ t+ O6 R% j1 k, Z$ e/ ?
- pthread_mutex_unlock(&pmt);
复制代码线程互斥锁使用之前需要初始化: - pthread_mutex_t pmt;+ p/ Q" E4 `- a% [) u. [
- pthread_cond_t pct;7 l+ a0 d" e$ J8 y, {5 U& i
- int main(int argc, char* argv[])
- d, n8 t8 Z* c0 j2 G D: v - {6 F. f# O4 k3 e& g9 W: ^8 ~
- .../ J" e) U9 L! O& W/ w& p
- TCP_Server_Found(&socket_web_server , (char*)argv[2] , PORT_TCP);
& v O8 n8 s* L9 `2 v* Y - pthread_mutex_init(&pmt , NULL);
6 S: R! r' h" w - pthread_create(&tid_tcp_web_recv , NULL , Thread_TCP_Web_Recv , NULL);. C& {( `: Z" N! R* z" z
- pthread_create(&tid_tcp_web_send , NULL , Thread_TCP_Web_Send , NULL);% N+ t& X8 H6 v" A' d8 c
- ...; A( v: e: D @( | K9 h# S
- while(1)
( l- M+ Y8 [% [0 s( J: K - {
. T) u8 d/ P4 m: j" r: B - V4l2_Grab_Mjpeg(false , MJPEG_FILE_NAME);
! G m' O& ?" t, f - ...6 O5 r l' u4 ^! m( ~
- }
9 l) ]9 }! ^4 h/ y - ...$ e* Q+ [% g' Y" K- g$ }$ S
- }
复制代码然后是发送的细节,发送图片文件之前,需要先发送HTTP标准头,这个相当于给发送图片或者其它类型的流数据铺路: - <p style="box-sizing: border-box; border: 0px; vertical-align: baseline; line-height: 26px;"><span style="text-indent: 32px;">
, n! n% y. u; G1 G3 M' h2 t# ` - </span></p><p style="box-sizing: border-box; border: 0px; vertical-align: baseline; line-height: 26px;"></p>
复制代码- #define STD_HEADER "Connection: close\r\n" \
/ x1 M7 R7 M( a+ F Z( N3 M# ? - "Server: MJPG-Streamer/0.2\r\n" \* Z) q# ]* n+ s
- "Cache-Control: no-store, no-cache, must-revalidate, pre-check=0, post-check=0, max-age=0\r\n" \
- R$ I: p! ?8 w; h# \2 h - "Pragma: no-cache\r\n" \
, T: a, s3 y9 Z4 B: |' `1 C - "Expires: Mon, 3 Jan 2000 12:34:56 GMT\r\n"" {6 W1 m& x; S8 U) l( w
- #define BOUNDARY "boundarydonotcross"0 X( w/ ^3 `/ v7 k7 Y
- printf("preparing header\n");2 K* c3 E: I6 m+ \+ F
- sprintf(buffer, "HTTP/1.0 200 OK\r\n" \8 X o2 H7 h1 \, s: @: K0 s5 k/ }
- "Access-Control-Allow-Origin: *\r\n" \
$ j- Z# M" F- L3 A: c* o1 V6 W - STD_HEADER \& U# U' s- E% m# u, y
- "Content-Type: multipart/x-mixed-replace;boundary=" BOUNDARY "\r\n" \$ `& y! q, J& j! Z/ G8 q& X/ d. H- o
- "\r\n" \
5 F+ Y. ^* c7 ^; U3 ?: t( b - "--" BOUNDARY "\r\n");# E6 h5 S6 j7 S( B1 j+ l* ]
- if(write(fd, buffer, strlen(buffer)) < 0)
! C9 h& Z; X! ]0 n7 A! `! K - {
7 B5 W+ e& \% z5 [4 }2 g - free(frame);
2 R. O( u' K" Z: {. K! e1 L - return;( V$ `9 [$ ]0 t% r( e
- }
复制代码发送完HTTP标准头之后,就需要发送内容头(Content-Type),这处的Content-Type为image/jpeg,同样,HTTP标准协议里面image支持的类型远不止jpeg一种,发送完内容头之后就是正文和boundary结尾,这样帧完整的HTTP头发送到指定的TCP GET地址,就会在浏览器中显示刚刚发送的图片: - <pre class="prettyprint lang-cpp" style="box-sizing: border-box; font-family: Monaco, Menlo, Consolas, "Courier New", monospace; font-size: 16px; white-space: pre-wrap; line-height: 1.38462; color: rgb(51, 51, 51); word-break: break-all; background-color: rgb(245, 245, 245); border: 0px; border-radius: 4px; vertical-align: baseline;"> sprintf(buffer, "Content-Type: image/jpeg\r\n" \
! N6 M7 m6 g5 Z4 l# a* T( r+ k - "Content-Length: %d\r\n" \6 E( s7 A4 t/ `& k/ d2 ~/ u. r
- "X-Timestamp: %d.%06d\r\n" \
. ?# l9 q1 G# P8 Q3 g" R - "\r\n", frame_size, (int)timestamp.tv_sec, (int)timestamp.tv_usec);7 }& S* E% u3 g) W& B& i
- printf("sending intemdiate header\n");
, w& M$ X$ V7 q - if(write(fd, buffer, strlen(buffer)) < 0)
8 T* w& q- O% j4 \' {- O& { - break;- T/ X5 e3 B9 k% m, t j4 k! p! f
- printf("sending frame\n");, A; y1 E' a" y0 o( i6 g
- if(write(fd, frame, frame_size) < 0) F9 n$ ~1 X' B" j
- break;
$ N" g N1 x9 O, }" n# t - printf("sending boundary\n");$ `3 @- W% r2 ]( K, M u* d0 H
- sprintf(buffer, "\r\n--" BOUNDARY "\r\n");
: [/ u* `0 F% N8 H% m - if(write(fd, buffer, strlen(buffer)) < 0). Z* [$ Q! R; a3 K E6 I
- break;</pre><p style="box-sizing: border-box; border: 0px; font-size: 16px; vertical-align: baseline; line-height: 26px; color: rgb(51, 51, 51); font-family: " helvetica="" neue",="" helvetica,="" tahoma,="" arial,="" "microsoft="" yahei",="" "hiragino="" sans="" gb",="" "wenquanyi="" micro="" hei",="" sans-serif;"=""></p>
复制代码另外需要说明的是,TCP服务器线程在发送MJPEG流的时候是死循环发送的,因此TCP客户端在发送完GET指令之后,就会收到TCP服务器循环发送的图像缓存,TCP客户端会陷入忙等待状态无法再对外发送任何GET或者POST指令,从客户端使用者角度来看的效果就是网页一直在等待。  8 R! o( V( s$ u. u- i9 T b6 @% D

p! }+ L2 [0 V4 K二、UDP上位机UDP发送操作,同样需要先建立UDP Socket:
" V! M+ w6 V$ M+ P- int UDP_Send_Found(socklen_t* socket_found , struct sockaddr_in *addr , char* ip , int port)
. L3 }& ~% e' S - {
+ y3 Q; n. x* l, V. Z$ O# s - *socket_found = socket(AF_INET, SOCK_DGRAM, 0);6 @7 [( I. _& D; u0 h7 A
- if(*socket_found == (~0))2 D3 }3 n% D T; M% Z) \& g
- {
* |( n7 F l1 b- ^& \2 a - printf("Create udp send socket failed!\n");
3 ^2 |8 l3 ^. J/ L; J - return -1;+ p+ h: j& \5 `! j6 [
- }
2 s' b6 a# A+ c: P' Q - addr->sin_family = AF_INET;/ [2 Q, P* r1 P4 T/ E
- addr->sin_addr.s_addr = inet_addr(ip);: K- K+ t5 ~- Z: O" k- K
- addr->sin_port = htons(port);
& l I0 b) |6 \ r% L0 B" _3 A* }0 l - memset(addr->sin_zero, 0, 8);
1 L7 m: ]. b+ y - return 0;
0 X: p+ D) Q) X5 Y% B J8 O - }
复制代码 4 ?, j! }% p* c; |& E% G
6 A8 R/ I3 b9 U
而UDP文件发送则要比HTTP发送简单得多,只需要将文件切片,每一片为固定长度的UDP帧长度,逐帧发送即可:
! Z( |+ ]0 l# i2 z: o
+ `2 n( _; {* z, v+ f. x1 M' `% M# m9 v& Z& b0 d! P
- while(fend > 0)7 h' M; q1 Y- Q) F3 c2 i3 F
- {$ D, ?, |" } [5 d7 s0 g0 I& D% [
- memset(picture.data , 0 , sizeof(picture.data));
u Z! B2 C x3 U' I5 E0 N* ^ - fread(picture.data , UDP_FRAME_LEN , 1, fp);+ e7 \0 h6 ~4 P, W% j
- if(fend >= UDP_FRAME_LEN)4 b, s, S) n' n" r7 z+ Y! b
- {
9 c+ X) A8 n$ r% { - picture.length = UDP_FRAME_LEN;
# z' v- L0 \6 \: T+ f2 B - picture.fin = 0;
) H/ T+ O2 k/ F, g7 F, U2 U - }& N* o: @* ]1 G& H) j- i
- else
' Z" H8 Z3 Z2 F) v+ @ - {+ S- o$ Q# a" G; D+ r [
- picture.length = fend;
5 B3 g+ Y/ p! D* Q% X - picture.fin = 1;& o5 @/ ?) W7 A
- }, H" l! \- Q2 O$ \0 W1 Y
- //printf("sendbytes = %d \n",sendbytes);
* t4 _( a2 b" i2 Y - sendbytes = sendto(socket_send, (char *)&picture, sizeof(struct Package), 0, (struct sockaddr*)&addr,addr_len);
9 A9 q' d7 l3 ~+ c - if(sendbytes == -1)/ q" Y, e" @- a3 `
- {& j. W1 @2 V5 o$ g- D4 i( X
- printf("Send Picture Failed!d\n");
) g5 o5 I4 l# ^" B { - return -1;
. d( c$ G1 f' M - }
' e; F# c% _- Y \% X - else
9 k9 v% n2 l+ f# ~+ O5 c6 \ - {
) g A% I& v* m9 c - fend -= UDP_FRAME_LEN;& q; W8 }2 L2 [$ J: Y" B% y
- }
) V9 L, i$ x- L - }
复制代码
* h/ c. ^! H+ } m3 [+ _, e: G% D c. [$ @
 . i) E2 M3 s8 O; Z9 `$ w7 |8 I
! m, P* l2 t E! t: M% fiMX8MPlus 核心板: https://www.forlinx.com/product/136.html |