本帖最后由 飞凌-marketing 于 2022-1-27 11:00 编辑 9 h6 g0 y, H: Q0 g8 |
1 H! F3 [' E$ y- m. s: l% n- j
 1 V' e4 a, K M7 \( p
作者|donatello1996 来源 | 电子发烧友 题图|飞凌嵌入式 iMX8MPlus 核心板: https://www.forlinx.com/product/136.html
1 w J( F% _/ ?) ?6 s* \; F
' Z" _0 d& r' z0 q; ~" e本文采用的硬件板卡为飞凌嵌入式OKMX8MP-C开发板,系统版本Linux5.4.70+Qt5.15.0,主要介绍基于HTTP网页服务器和UDP上位机的MJPG码流传输。 MJPG格式作为一种持续传输的视频码流,在远程监控领域中应用较广,而实现这种远程监控的第三方应用最常见的有两种:浏览器HTTP网页、UDP上位机。 6 v0 `/ I5 y* S* t2 m! h
 " O6 V+ ~: p9 F1 t8 j: X
两者各有优势,对比鲜明,其中: 这两种应用各有优缺点,对于嵌入式开发者来说,两者都必须掌握。
6 c5 L9 S; ?4 l4 _7 L一、HTTP网页服务器2 E- o3 I- W b2 P A9 \9 u) J
先说下HTTP网页服务器获取MJPG码流的代码,首先是OKMX8MP-C在开发板端建立TCP服务器: - int TCP_Server_Found(socklen_t* socket_found , char* ip , int port); e' _- P- [- x& u9 x8 w
- {' o, }/ Z4 }8 |7 ~% Q; X) _% J3 f
- struct sockaddr_in servaddr;
& B6 x3 }/ M# l- J$ @ - socklen_t addrsize = sizeof(struct sockaddr);
$ g9 [1 w& C: P' n+ `; I - bzero(&servaddr , sizeof(servaddr));/ G" I5 @! l8 T. `4 B |
- servaddr.sin_family = AF_INET;9 s4 [: e" K+ x+ S% M0 `0 F
- servaddr.sin_addr.s_addr = inet_addr(ip);
* Q, ~. w; J6 T! @8 z+ H/ m2 K - servaddr.sin_port = htons(port);
+ v8 N% N6 n6 [. P - int ret;
8 Q% C* a v8 |" Y" u - IF( (*socket_found = socket(AF_INET , SOCK_STREAM , 0)) == -1)5 x( F4 o5 u5 b4 e& `' Q
- {" _5 o7 D7 X2 O- }& X1 F/ Z. q
- printf("Create socket error: %s (errno :%d)\n",strerror(errno),errno);
# j ^- n1 d5 i: p- Q$ K - return -1;" ^ H# o* b& a4 I1 ?3 @0 c1 q
- }- O- p1 H: E, j0 q) }4 N$ k
- int on = 1;8 y3 j' @. c3 ?: l- {
- if(setsockopt(*socket_found , SOL_SOCKET, SO_REUSEADDR, &on, sizeof(on)) < 0)
( }: K: d) K6 d) C0 N2 y3 e3 ~ - {
+ @4 ~3 a m R4 u# K5 H1 Y - printf("setsockopt error\n");
$ B" Y2 g. J) I! F - }
$ x' ^/ ~* h$ I6 }+ } - ret = bind(*socket_found , (struct sockaddr *)&servaddr , addrsize);( S4 S% ]7 V& q6 h y' p
- if(ret == -1)& h; H7 u' }6 @$ Z' F
- {* [9 A" J k6 I( h" ]
- printf("Tcp bind faiLED!\n");0 S9 u3 ?0 C) {
- return -1;
2 `( Q3 B7 `/ J$ h9 X z - }
; z3 i1 n& G7 M' w0 [ K& J4 M" E - if(listen(*socket_found , 5) == -1)4 G6 S- d8 X& j: t6 N& L
- {
) N* Y2 Z0 d$ v4 @ - printf("Listen failed!\n");
. q d6 G) L0 n4 U- F' f' D) a - return -1;7 J) _/ w3 h$ b1 d- o$ k8 y
- }
3 j. N9 Z! T5 ?, v$ A J - return 0;: q6 w; F& k a, [! A
- }
复制代码其中setsockopt()函数是可选的,一般只用于规避socket()函数的建立错误。 建立了TCP服务器后,返回的socklen_t型实参在后面的HTTP网页服务器中需要用到。 HTTP网页服务器所属的TCP操作是需要另起轮询线程来让客户端进行accept()握手操作的,accept()之前的listen()倒是只需要执行一次即可,accept()握手操作和recv()接收操作需要创建一个死循环线程: - pthread_create(&tid_tcp_web_recv , NULL , Thread_TCP_Web_Recv , NULL);
' }( Y M9 F, A, G* Z0 }; F - void * Thread_TCP_Web_Recv(void *arg)) P( Y% c8 \# n0 }- {# o# l
- {
$ L, ]$ P9 C8 I7 T# w6 V' N' B9 O+ k$ ~ - 。。。! w) l, ~ _' }: \& [7 c0 l1 P/ j
- while(1)/ M/ i+ O" g* w9 ^( j. J' B# }
- {* h+ s$ G+ P3 D+ @1 J: m, D
- fd_socket_conn = accept(socket_web_server , (struct sockaddr *)&sockaddr_in_conn , &addrsize);8 C7 t2 \/ o: s+ R
- printf("fd_socket_conn = accept()\n");
y8 M% }9 o% v; w - 。。。* v2 l' R$ s! b: z
- recv(fd_socket_conn , recvbuf , 1000 , 0);/ r6 F0 ]0 v: U: h: q
- }
7 _8 l) \: n# ^ - 。。。9 k8 [+ Y) I' k& G' \" B3 Q
- }
复制代码MJPG帧可以使用Grab操作获取,获取到的MJPG帧需要在TCP线程中读,在Grab操作线程中写,这种被多个线程访问的资源需要加锁防止读写冲突,即资源被Grab操作写入时,需要上锁,不允许其它线程访问,操作完成时需要解锁,允许其它线程访问: - pthread_mutex_lock(&pmt);6 i! m- t" t( k4 N' y; _7 H
- pic_tmpbuffer = pic.tmpbuffer;$ e3 d" X0 {$ L6 q0 U7 E9 w
- pic.tmpbytesused = buff.bytesused;* U$ J/ I5 d, }6 }% r. \' N$ @1 d
- pic_tmpbytesused = pic.tmpbytesused;+ n) ~7 i' V; @
- pthread_cond_broadcast(&pct);9 C3 O0 C% l7 N- w% y9 a
- pthread_mutex_unlock(&pmt);
复制代码线程互斥锁使用之前需要初始化: - pthread_mutex_t pmt;5 J8 w& A2 T: p; r; C
- pthread_cond_t pct;; X4 G" h. {1 r4 [4 T1 g1 a6 E
- int main(int argc, char* argv[])
/ q$ N p, w/ _ - {
& ^4 @ _3 w# N8 S, E - ...
# Q( v$ Q4 n# j% u4 G - TCP_Server_Found(&socket_web_server , (char*)argv[2] , PORT_TCP);9 V# E; z w% k- {1 I: U
- pthread_mutex_init(&pmt , NULL);# ?5 I: W: c6 L, u( u4 Z" Z _
- pthread_create(&tid_tcp_web_recv , NULL , Thread_TCP_Web_Recv , NULL);
- L! l5 O3 W Y' U/ Q& h Y - pthread_create(&tid_tcp_web_send , NULL , Thread_TCP_Web_Send , NULL);
. n& `1 H7 m6 o& o' c7 G8 W7 ?7 E% z - ...
& B8 k' u o( i% O - while(1)
4 |0 b+ l) D$ }$ G1 _2 \/ ? - {
v- f v. i6 Q- x+ O) Z4 u& |& n' `3 ^ - V4l2_Grab_Mjpeg(false , MJPEG_FILE_NAME);
5 d7 f, Z9 ~- g( G! |7 B - ...) U" I x, ^2 U1 M
- }8 ?1 ^# E O! a- g: M1 m
- ...
& Q8 `% }* W$ J) `! c - }
复制代码然后是发送的细节,发送图片文件之前,需要先发送HTTP标准头,这个相当于给发送图片或者其它类型的流数据铺路: - <p style="box-sizing: border-box; border: 0px; vertical-align: baseline; line-height: 26px;"><span style="text-indent: 32px;">
+ Z8 M& I* p9 u; d4 s - </span></p><p style="box-sizing: border-box; border: 0px; vertical-align: baseline; line-height: 26px;"></p>
复制代码- #define STD_HEADER "Connection: close\r\n" \0 G0 J* M5 L5 K; R
- "Server: MJPG-Streamer/0.2\r\n" \
. }, x) A4 s3 k$ Z - "Cache-Control: no-store, no-cache, must-revalidate, pre-check=0, post-check=0, max-age=0\r\n" \. p+ S! H7 r8 K0 ^
- "Pragma: no-cache\r\n" \7 C4 t. Y7 m1 B3 G2 d6 I' v) E" c
- "Expires: Mon, 3 Jan 2000 12:34:56 GMT\r\n"
& J- X* E) d5 V W* p - #define BOUNDARY "boundarydonotcross"
" g ?2 w; H k4 Z( F - printf("preparing header\n");
9 ^) ~: O; a3 s2 _3 g - sprintf(buffer, "HTTP/1.0 200 OK\r\n" \: H9 v7 d( C8 j4 @; R
- "Access-Control-Allow-Origin: *\r\n" \
8 f3 D* F7 K3 R& G) l7 J& F - STD_HEADER \
- R1 x3 i( Z0 l" M% |9 _ - "Content-Type: multipart/x-mixed-replace;boundary=" BOUNDARY "\r\n" \, [9 e5 c6 C4 k1 H# m: u4 ~
- "\r\n" \
+ u5 d x n8 R& J& n& h# s) ^ - "--" BOUNDARY "\r\n");
B& B6 b: Y% g/ a% {% x. Y - if(write(fd, buffer, strlen(buffer)) < 0)
) S" C4 J7 |: x - {5 |0 W7 A) V z& ?5 P/ U
- free(frame);) A9 w1 c8 l( U% Q- l! R
- return;
7 J' t. b/ x0 Y8 W* d$ b - }
复制代码发送完HTTP标准头之后,就需要发送内容头(Content-Type),这处的Content-Type为image/jpeg,同样,HTTP标准协议里面image支持的类型远不止jpeg一种,发送完内容头之后就是正文和boundary结尾,这样帧完整的HTTP头发送到指定的TCP GET地址,就会在浏览器中显示刚刚发送的图片: - <pre class="prettyprint lang-cpp" style="box-sizing: border-box; font-family: Monaco, Menlo, Consolas, "Courier New", monospace; font-size: 16px; white-space: pre-wrap; line-height: 1.38462; color: rgb(51, 51, 51); word-break: break-all; background-color: rgb(245, 245, 245); border: 0px; border-radius: 4px; vertical-align: baseline;"> sprintf(buffer, "Content-Type: image/jpeg\r\n" \, c) F7 F" l0 }# w. ` s5 Z R( x
- "Content-Length: %d\r\n" \9 g, A9 ]9 I' c
- "X-Timestamp: %d.%06d\r\n" \
* q' r- {: B u* ] - "\r\n", frame_size, (int)timestamp.tv_sec, (int)timestamp.tv_usec);
# c# G: ?+ X( G! y. x - printf("sending intemdiate header\n");
+ _$ K8 Q o% Z! P - if(write(fd, buffer, strlen(buffer)) < 0)! i' `2 l! y9 s* Q8 t9 S
- break;
) X* n# M# H- ` r+ s3 R# b - printf("sending frame\n");5 R5 T7 [' Q7 { j Y, A5 c) Q( P
- if(write(fd, frame, frame_size) < 0)
; g! c: J3 D0 v# J - break; { M2 }, d! q6 E1 B9 k/ C
- printf("sending boundary\n");
' {2 N8 ^8 v: T) z' \% B - sprintf(buffer, "\r\n--" BOUNDARY "\r\n");
7 a9 P! J ]9 }0 Z' T& j - if(write(fd, buffer, strlen(buffer)) < 0)
! m$ w7 t! W! d j+ b; M - break;</pre><p style="box-sizing: border-box; border: 0px; font-size: 16px; vertical-align: baseline; line-height: 26px; color: rgb(51, 51, 51); font-family: " helvetica="" neue",="" helvetica,="" tahoma,="" arial,="" "microsoft="" yahei",="" "hiragino="" sans="" gb",="" "wenquanyi="" micro="" hei",="" sans-serif;"=""></p>
复制代码另外需要说明的是,TCP服务器线程在发送MJPEG流的时候是死循环发送的,因此TCP客户端在发送完GET指令之后,就会收到TCP服务器循环发送的图像缓存,TCP客户端会陷入忙等待状态无法再对外发送任何GET或者POST指令,从客户端使用者角度来看的效果就是网页一直在等待。  4 R- V4 k: e' J: d' J( y8 Q7 E

7 j5 |. Q; H# ?$ |3 W6 T+ ?( m& p二、UDP上位机UDP发送操作,同样需要先建立UDP Socket:* v2 f6 N/ o; a$ R3 r1 O
- int UDP_Send_Found(socklen_t* socket_found , struct sockaddr_in *addr , char* ip , int port)
J9 ]% h! Y$ x - {) z. r4 t: e- ?. I4 }
- *socket_found = socket(AF_INET, SOCK_DGRAM, 0);8 [; E) R# f4 k, a! m+ S6 i6 B
- if(*socket_found == (~0))9 G3 o& @5 y5 _) t. X6 p* L
- {
0 ?; C$ p, w6 u$ A; f1 d' S* O - printf("Create udp send socket failed!\n");7 T( y5 y8 S$ C8 Y1 s2 [
- return -1;
( N: y4 m& g% C: @1 c1 o - }( A8 z; ]2 ^, c4 k& w" D
- addr->sin_family = AF_INET;
" z& c9 n8 z2 c, v7 l/ Q - addr->sin_addr.s_addr = inet_addr(ip);
9 {2 q* X: a" T9 w; @ Q - addr->sin_port = htons(port);3 W# r, a! o0 L
- memset(addr->sin_zero, 0, 8);
}9 B! D, C3 u" I - return 0; v& m( Q6 Y; a4 b% u# b6 y8 o2 d$ v
- }
复制代码 & f E. v& g0 F! k
" d) k$ |0 v6 D# Q# i+ ?" V0 q
而UDP文件发送则要比HTTP发送简单得多,只需要将文件切片,每一片为固定长度的UDP帧长度,逐帧发送即可:
+ ]0 |" H/ ~/ \8 m, d# S6 l# d; u
- P: U' V, \/ m; r6 @. C0 a4 w1 B$ U+ x
- while(fend > 0)
/ ?. h* H& h0 u3 W/ s - {
; v* m6 y1 ]+ t+ K - memset(picture.data , 0 , sizeof(picture.data));
& p1 a! ]) X" G$ L/ e1 m- y - fread(picture.data , UDP_FRAME_LEN , 1, fp);, x/ W+ Z+ q5 y* J
- if(fend >= UDP_FRAME_LEN)
# @: l& r9 ?" ?( b) v3 M - {2 J" g. ?7 ` x( ~
- picture.length = UDP_FRAME_LEN;
. s7 F0 d; {3 O5 J; V( t - picture.fin = 0;8 W- j$ |7 j; ?% Z) u' d. w
- }% F5 ~3 X9 F! j5 ~$ Y
- else
# s: u; D4 N# G7 w - {# Q9 Q( v! q; i/ C- T
- picture.length = fend;/ H$ e4 G \6 _5 P }! e
- picture.fin = 1;. @1 _! S+ h7 U) b
- }/ S4 Q. x3 A! v3 w, y0 k& m
- //printf("sendbytes = %d \n",sendbytes);
% F. Q/ p& S: \8 _ - sendbytes = sendto(socket_send, (char *)&picture, sizeof(struct Package), 0, (struct sockaddr*)&addr,addr_len);+ b+ d7 X% Y- g3 c
- if(sendbytes == -1)+ \5 H6 r$ P2 N$ q, h
- {/ k7 k$ T, T/ L1 U0 P% E
- printf("Send Picture Failed!d\n");
/ q9 ]( `* @! H' C0 }4 b - return -1;! W8 L5 `$ Z3 I; h& }
- }
. M' l/ X1 K5 V1 a# t6 \1 o - else* x5 i' L6 z2 P8 a: U2 ^
- {; z: g( }: D2 G
- fend -= UDP_FRAME_LEN;
: k& Q4 B [' w1 K - }* A0 P- `, W J9 ]: {! D
- }
复制代码
& L6 v& E- M0 G2 K* T* W, J6 ]) N$ F% U$ Y0 |0 y2 s) j

* t" a s( ?: \8 f* ~/ k8 E1 ~, n% f$ I, K+ C8 V
iMX8MPlus 核心板: https://www.forlinx.com/product/136.html |