本帖最后由 飞凌-marketing 于 2022-1-27 11:00 编辑 4 x+ e# {; W6 r5 a/ }7 w! I
. u* b+ s: \1 f1 v; x0 f

5 X$ t9 B6 V, T( J作者|donatello1996 来源 | 电子发烧友 题图|飞凌嵌入式 iMX8MPlus 核心板: https://www.forlinx.com/product/136.html. a' p% u) E! q4 G4 ?: h3 b7 L% p
/ n4 ~' `# ]- |0 M! k3 X本文采用的硬件板卡为飞凌嵌入式OKMX8MP-C开发板,系统版本Linux5.4.70+Qt5.15.0,主要介绍基于HTTP网页服务器和UDP上位机的MJPG码流传输。 MJPG格式作为一种持续传输的视频码流,在远程监控领域中应用较广,而实现这种远程监控的第三方应用最常见的有两种:浏览器HTTP网页、UDP上位机。
& b5 a- t$ ~5 w4 w& E1 n4 t* v8 Q& F) i . z& c2 v/ i9 R
两者各有优势,对比鲜明,其中: 这两种应用各有优缺点,对于嵌入式开发者来说,两者都必须掌握。
$ m- U3 _* Y O* ]5 g- V- H一、HTTP网页服务器& a4 [8 I& s2 L2 K- _) q
先说下HTTP网页服务器获取MJPG码流的代码,首先是OKMX8MP-C在开发板端建立TCP服务器: - int TCP_Server_Found(socklen_t* socket_found , char* ip , int port)" ]# S) J+ ?7 d! `
- {
, G h' s- A" }# G* _$ ~ - struct sockaddr_in servaddr;
5 `, Y1 h+ p7 V - socklen_t addrsize = sizeof(struct sockaddr);1 X6 \: f6 y% m) R: h) c
- bzero(&servaddr , sizeof(servaddr));
" e- d. G& W& B3 R1 u6 X$ [ - servaddr.sin_family = AF_INET;
- |, z% f0 u' f* `& x7 ]' k" P$ L - servaddr.sin_addr.s_addr = inet_addr(ip);
% g* l, J, y8 A: A7 m - servaddr.sin_port = htons(port);4 B/ I4 d/ F8 z( p( Z4 m: e& f
- int ret;; W s* ?1 I0 ?/ J1 A; B9 i! c" w
- IF( (*socket_found = socket(AF_INET , SOCK_STREAM , 0)) == -1)
h/ y, K6 |' m: K - {
6 J- f1 q8 A, q, A7 O1 n - printf("Create socket error: %s (errno :%d)\n",strerror(errno),errno);/ h+ c/ Y9 D. y. ^! l3 Z
- return -1; D9 E0 J' x5 v6 T' T* U, _9 Q
- }
5 Z& b& X' x. s! p7 G. Y - int on = 1;( V& n: c* a% U. {4 j9 x
- if(setsockopt(*socket_found , SOL_SOCKET, SO_REUSEADDR, &on, sizeof(on)) < 0)6 L5 l/ E L% ?: G3 g) V5 T4 w
- {3 M$ @1 G) }& ] t
- printf("setsockopt error\n");
9 n% c( ^/ }5 L0 t4 Z T2 u. | - }+ Q) b. _; P$ b& T: Y; A
- ret = bind(*socket_found , (struct sockaddr *)&servaddr , addrsize);
1 l; q3 L" Z, q6 b0 ? - if(ret == -1)
O6 f( m' U+ V, K - {
+ \6 {; d$ L4 ]! a# n. \ - printf("Tcp bind faiLED!\n");. Z, k5 j. m$ z5 t
- return -1;) f& d% ~ z2 N: h/ M3 ~
- }: b4 u. Z/ x& L9 q; Z
- if(listen(*socket_found , 5) == -1)
" q& |7 z. N6 j5 f - {5 x" x# D F6 F& h& D. g
- printf("Listen failed!\n");
* `+ [# ?6 |1 `' X& {3 r( E* o; b - return -1;
2 U3 U7 j) Y/ i/ i& G; r - }4 l, ^3 N1 D: h. k
- return 0;/ f E" t6 l& b/ j7 Z3 V% ~2 O
- }
复制代码其中setsockopt()函数是可选的,一般只用于规避socket()函数的建立错误。 建立了TCP服务器后,返回的socklen_t型实参在后面的HTTP网页服务器中需要用到。 HTTP网页服务器所属的TCP操作是需要另起轮询线程来让客户端进行accept()握手操作的,accept()之前的listen()倒是只需要执行一次即可,accept()握手操作和recv()接收操作需要创建一个死循环线程: - pthread_create(&tid_tcp_web_recv , NULL , Thread_TCP_Web_Recv , NULL); V) o2 m% s% J1 o, h
- void * Thread_TCP_Web_Recv(void *arg)
. \: Y/ R) M8 {9 g. c6 D - {7 Y0 m1 L/ l- X& B6 ^5 w/ S# h' r
- 。。。
7 t, v: i/ K l- E( ?, t% j - while(1), [/ G j0 ~7 t6 d" S5 m
- {
) }! I+ T) w3 ]7 _- g$ @- w) p - fd_socket_conn = accept(socket_web_server , (struct sockaddr *)&sockaddr_in_conn , &addrsize);2 c9 u3 q7 U- t; W: G# @5 |
- printf("fd_socket_conn = accept()\n");
" C7 ]- Z$ i9 l- E - 。。。1 X9 J) S$ g1 B3 t2 r
- recv(fd_socket_conn , recvbuf , 1000 , 0);% X j# X. s" z" A5 {8 H" C, G
- }
+ v: m' a. B! r$ @4 H3 A- [ - 。。。0 {& ~/ q$ r' O$ y, @& f
- }
复制代码MJPG帧可以使用Grab操作获取,获取到的MJPG帧需要在TCP线程中读,在Grab操作线程中写,这种被多个线程访问的资源需要加锁防止读写冲突,即资源被Grab操作写入时,需要上锁,不允许其它线程访问,操作完成时需要解锁,允许其它线程访问: - pthread_mutex_lock(&pmt);
0 U- ]/ D, u* P% a! J - pic_tmpbuffer = pic.tmpbuffer;
* d, b- v$ b0 f, T0 C - pic.tmpbytesused = buff.bytesused;9 I9 i0 q, Z+ q9 a3 V* v4 x7 _
- pic_tmpbytesused = pic.tmpbytesused;5 Q) V1 _5 J& g
- pthread_cond_broadcast(&pct);
z& ?7 H: g7 k+ z# U - pthread_mutex_unlock(&pmt);
复制代码线程互斥锁使用之前需要初始化: - pthread_mutex_t pmt;
+ ]2 a& j$ w3 \6 y2 q: H4 ]3 C - pthread_cond_t pct;1 |, J0 J* _4 k+ V: A! A
- int main(int argc, char* argv[]); w, T" b! p" k. U% {' X3 p
- {* L1 e- s8 W8 R8 p' |! z
- .... y8 Z1 I/ s1 I8 r# n5 h9 r4 k; n4 I
- TCP_Server_Found(&socket_web_server , (char*)argv[2] , PORT_TCP);
# O9 D6 _( L* l* D, @ - pthread_mutex_init(&pmt , NULL);* O# w. O3 s: r; T
- pthread_create(&tid_tcp_web_recv , NULL , Thread_TCP_Web_Recv , NULL);% V+ u$ D; o/ h9 P
- pthread_create(&tid_tcp_web_send , NULL , Thread_TCP_Web_Send , NULL);% ]2 A" t6 y6 b; V( y& K7 b
- ...
: S2 H3 Z3 r9 o, [) F( x - while(1) V8 [' M8 L0 Q3 r4 T
- {" C0 Q- v; L; B7 t
- V4l2_Grab_Mjpeg(false , MJPEG_FILE_NAME);
' M: {4 {4 v$ u3 n L - ...( T: f8 z* L% ]* ? \
- }
. b/ I9 `2 U% b" n3 M - ...
+ a5 q% X, I9 w! E2 v$ z2 i! P - }
复制代码然后是发送的细节,发送图片文件之前,需要先发送HTTP标准头,这个相当于给发送图片或者其它类型的流数据铺路: - <p style="box-sizing: border-box; border: 0px; vertical-align: baseline; line-height: 26px;"><span style="text-indent: 32px;">
; \; _1 Y+ `& }- E4 s/ B+ m - </span></p><p style="box-sizing: border-box; border: 0px; vertical-align: baseline; line-height: 26px;"></p>
复制代码- #define STD_HEADER "Connection: close\r\n" \
* k- k5 n/ h4 g! p# V - "Server: MJPG-Streamer/0.2\r\n" \( L5 `/ f. e/ }% M- F, k' p0 v
- "Cache-Control: no-store, no-cache, must-revalidate, pre-check=0, post-check=0, max-age=0\r\n" \5 {4 m! j9 Q& C; u n( B( C7 Z
- "Pragma: no-cache\r\n" \8 @/ l# a* e* J" N8 V4 Y$ t
- "Expires: Mon, 3 Jan 2000 12:34:56 GMT\r\n"
/ C- U$ {6 c+ f- Z* Y3 h" ~7 z) x - #define BOUNDARY "boundarydonotcross"* N$ d/ d' Z3 _* `) X
- printf("preparing header\n");: L8 I' K3 J; v# d9 I
- sprintf(buffer, "HTTP/1.0 200 OK\r\n" \, q ^- L$ w e6 ^( k; O
- "Access-Control-Allow-Origin: *\r\n" \
& b r, @ t! j1 {8 E - STD_HEADER \' x! Z' A% d2 G+ g Z( i9 Y
- "Content-Type: multipart/x-mixed-replace;boundary=" BOUNDARY "\r\n" \
0 j1 q! H* c* g: `. u5 d! h - "\r\n" \ P! R+ e4 y" k7 K" _ K
- "--" BOUNDARY "\r\n");
* u; t5 k: g9 R2 Z# x! K$ t - if(write(fd, buffer, strlen(buffer)) < 0)$ z9 N" v& Y6 z2 L6 \. c6 i
- {8 J: v: K1 ~- ]& B3 f& L
- free(frame);
: a% V& P, }; N! A5 J - return;
& ]$ n E2 w( }' m) q6 Q$ i8 Y7 M4 F - }
复制代码发送完HTTP标准头之后,就需要发送内容头(Content-Type),这处的Content-Type为image/jpeg,同样,HTTP标准协议里面image支持的类型远不止jpeg一种,发送完内容头之后就是正文和boundary结尾,这样帧完整的HTTP头发送到指定的TCP GET地址,就会在浏览器中显示刚刚发送的图片: - <pre class="prettyprint lang-cpp" style="box-sizing: border-box; font-family: Monaco, Menlo, Consolas, "Courier New", monospace; font-size: 16px; white-space: pre-wrap; line-height: 1.38462; color: rgb(51, 51, 51); word-break: break-all; background-color: rgb(245, 245, 245); border: 0px; border-radius: 4px; vertical-align: baseline;"> sprintf(buffer, "Content-Type: image/jpeg\r\n" \ S( D2 e m1 X- n7 U
- "Content-Length: %d\r\n" \' C+ ?, S, e: j. A* N/ h
- "X-Timestamp: %d.%06d\r\n" \& ^) }8 O3 V& Z% _% a/ T# r( u" R
- "\r\n", frame_size, (int)timestamp.tv_sec, (int)timestamp.tv_usec);
$ @7 m t0 B2 _, N0 Q3 r - printf("sending intemdiate header\n");
; a# {% Y8 l7 }, m5 T9 b5 A - if(write(fd, buffer, strlen(buffer)) < 0)
- J- t, u, }5 ^* n+ o% ~ - break;# y( x+ p! e5 s/ \( ~4 Z/ x
- printf("sending frame\n");% o. `/ r6 g5 n' Q; I1 P" F
- if(write(fd, frame, frame_size) < 0)
$ }6 R* B+ T0 h/ b( X, v - break;( K( \4 S, E$ o1 l
- printf("sending boundary\n");5 }8 C( C; t: q" _9 ]( Z/ ^/ g
- sprintf(buffer, "\r\n--" BOUNDARY "\r\n");
# |+ c8 t7 P9 R& H - if(write(fd, buffer, strlen(buffer)) < 0)
6 }2 |. }2 J; |2 R - break;</pre><p style="box-sizing: border-box; border: 0px; font-size: 16px; vertical-align: baseline; line-height: 26px; color: rgb(51, 51, 51); font-family: " helvetica="" neue",="" helvetica,="" tahoma,="" arial,="" "microsoft="" yahei",="" "hiragino="" sans="" gb",="" "wenquanyi="" micro="" hei",="" sans-serif;"=""></p>
复制代码另外需要说明的是,TCP服务器线程在发送MJPEG流的时候是死循环发送的,因此TCP客户端在发送完GET指令之后,就会收到TCP服务器循环发送的图像缓存,TCP客户端会陷入忙等待状态无法再对外发送任何GET或者POST指令,从客户端使用者角度来看的效果就是网页一直在等待。  1 V3 D# H- r9 A# I* n, [! [% ~

0 u. H$ N5 M6 O% X1 z二、UDP上位机UDP发送操作,同样需要先建立UDP Socket:3 k% v9 {+ e" Z
- int UDP_Send_Found(socklen_t* socket_found , struct sockaddr_in *addr , char* ip , int port)9 F5 A" w$ O+ c. M' h3 p3 h
- {9 k- O0 D% `" Q; E, w
- *socket_found = socket(AF_INET, SOCK_DGRAM, 0);
. c: F& t" l' W7 e - if(*socket_found == (~0))
+ J7 E, w: {* k% {0 m - {
$ k: Q# Z8 D9 T; A0 ~# ^ - printf("Create udp send socket failed!\n");1 W' r1 p" T# K M' r, F7 h1 l
- return -1;
, E8 n/ k% H9 H9 v - }
! p' p+ q, m) ? a& @( b - addr->sin_family = AF_INET;
% [1 O! e! o8 x) }( z. Q - addr->sin_addr.s_addr = inet_addr(ip);
! ?- H# V4 k- g# t5 \$ v% ~ - addr->sin_port = htons(port);
8 G- @1 T- U4 ]9 a; b; R$ s- \ - memset(addr->sin_zero, 0, 8);
; f0 f9 z$ |4 i) V4 e9 U - return 0;
& u" G2 }8 }& ]8 S: K - }
复制代码
$ ? d- D3 Y9 y4 j ]* h0 J' m, V" Y; Z3 U; W( P" }6 G
而UDP文件发送则要比HTTP发送简单得多,只需要将文件切片,每一片为固定长度的UDP帧长度,逐帧发送即可:$ }3 w$ o- m0 f2 I
- q- S+ E4 y) Z* R6 m5 k7 t
8 Y9 b. X# _/ I
- while(fend > 0)2 O: v5 l) A F; Y9 `
- {
' s) \7 T: p% Z3 l/ q$ ^+ N! g5 S - memset(picture.data , 0 , sizeof(picture.data));
& j( G. j( L7 O) s% S - fread(picture.data , UDP_FRAME_LEN , 1, fp);2 h0 V/ [( B! J! N* ]6 u
- if(fend >= UDP_FRAME_LEN)9 N; v5 e9 Z9 u8 N- H6 d8 e D
- {1 V; i1 @% F6 q6 f
- picture.length = UDP_FRAME_LEN;2 R. i! J( u5 S# ?% s9 }1 e/ z% E
- picture.fin = 0;
5 X* ?# C; G @5 n7 z7 ]% j: x* { - }6 a% N& I ^3 n9 P: i- X
- else% [( N6 c% p; C& @
- {/ a) F* N' r+ W2 S2 Q3 l
- picture.length = fend;9 N# Z" e5 P) p2 p) J9 \2 q
- picture.fin = 1;
|8 W0 G. ], m, o - }! t& k0 N" E, N) {
- //printf("sendbytes = %d \n",sendbytes);
r& ~4 ^) M, E+ }& j1 k - sendbytes = sendto(socket_send, (char *)&picture, sizeof(struct Package), 0, (struct sockaddr*)&addr,addr_len);- c/ I. B6 U4 \) Q+ N
- if(sendbytes == -1)! o& [+ ~! D& W
- {5 y7 \$ {1 @0 q' J( _6 J
- printf("Send Picture Failed!d\n");
( ~1 p T+ t3 n* d$ D - return -1;9 u4 s' z7 d: [# B2 E8 a
- }
4 h5 k6 p0 |6 N- u) F - else
$ e1 X/ m9 E3 n% L2 G1 y4 w% k; U - {
' V i. k' @1 Q! c% A: j6 d) H7 M - fend -= UDP_FRAME_LEN;
' A# @! v# \* p* d+ Z. W; |& M6 { - }5 r" E4 g. q1 Q& Q% J+ ?
- }
复制代码
6 l/ r% v3 |) C1 V2 E/ _5 Y: e0 o5 t3 P, g m
 9 u0 ~, ?1 n0 y- f% O( w, r
/ h% A; U& l# _
iMX8MPlus 核心板: https://www.forlinx.com/product/136.html |