本帖最后由 飞凌-marketing 于 2022-1-27 11:00 编辑
' v8 m7 j+ Z" k0 W' i, c
" ?. ]2 C! Y5 v3 H, ? 2 Z. [0 c' e% \) ] h( Y+ G
作者|donatello1996 来源 | 电子发烧友 题图|飞凌嵌入式 iMX8MPlus 核心板: https://www.forlinx.com/product/136.html; U0 n6 r1 D" m# W
5 w) \; @- K9 Z
本文采用的硬件板卡为飞凌嵌入式OKMX8MP-C开发板,系统版本Linux5.4.70+Qt5.15.0,主要介绍基于HTTP网页服务器和UDP上位机的MJPG码流传输。 MJPG格式作为一种持续传输的视频码流,在远程监控领域中应用较广,而实现这种远程监控的第三方应用最常见的有两种:浏览器HTTP网页、UDP上位机。
0 `9 ]3 U# w2 x4 g$ w4 p2 X" l# g8 A ) H) T9 y$ h1 K' q: ^" |) u; h; M
两者各有优势,对比鲜明,其中: 这两种应用各有优缺点,对于嵌入式开发者来说,两者都必须掌握。
2 w( C0 V8 m' h9 D! h. O一、HTTP网页服务器! L7 v( G# {5 I3 ~, V
先说下HTTP网页服务器获取MJPG码流的代码,首先是OKMX8MP-C在开发板端建立TCP服务器: - int TCP_Server_Found(socklen_t* socket_found , char* ip , int port)
& Y2 r& Y3 D: J# [7 L - {# E5 P; I: X( ^- U5 H
- struct sockaddr_in servaddr;
7 R- @9 B# O# O* _7 J3 G, { - socklen_t addrsize = sizeof(struct sockaddr);
6 R4 Y6 b3 q+ o+ c( O, R' I- X - bzero(&servaddr , sizeof(servaddr));' Q5 b a. J1 y2 w
- servaddr.sin_family = AF_INET;2 D% r, C w/ l. N; x
- servaddr.sin_addr.s_addr = inet_addr(ip);
2 ^% s2 f' B) x) B - servaddr.sin_port = htons(port);
& w' q+ E$ Z6 E& r+ d - int ret;( Y5 q6 n. e; ]- }5 V6 k; |* |+ C
- IF( (*socket_found = socket(AF_INET , SOCK_STREAM , 0)) == -1)
( M \/ z @; O+ Q9 H - {- u+ m3 r n N
- printf("Create socket error: %s (errno :%d)\n",strerror(errno),errno);
) v1 P. c) y- A - return -1;
$ ~4 T- Z% B& u) A5 m$ `: V - }, z$ V3 M" M0 _9 w3 S
- int on = 1;4 g1 ^" j! z; y5 A7 ]* w4 ?3 ^7 X
- if(setsockopt(*socket_found , SOL_SOCKET, SO_REUSEADDR, &on, sizeof(on)) < 0)* T H/ N$ `; O2 N9 p
- {* d' ~: \. i% c1 J% S
- printf("setsockopt error\n");
- A2 v, o8 E- W; b - }* ?3 v8 t* _0 k% c' w$ R
- ret = bind(*socket_found , (struct sockaddr *)&servaddr , addrsize);9 }; e% `6 d* T0 }# w- I+ ?# r
- if(ret == -1)
, B( @0 {3 |6 W9 A; V; h" s - {" g! I) B& _: x/ F; l
- printf("Tcp bind faiLED!\n");7 L. S1 \5 f- c) O% D
- return -1;6 j: R+ d4 ?1 D# D" h
- }7 X) I {/ \( F
- if(listen(*socket_found , 5) == -1)9 {8 ]- ^4 k3 x' f' K
- {
0 K1 `: ~: o- Y; d5 T: r( H& ] - printf("Listen failed!\n");
; w/ v2 }' n" `: a - return -1;
1 w& u0 I q2 Y0 M- ] - }
0 G( F, h" Y) _% r* ~" a/ X - return 0;
1 D5 M! G }9 ] - }
复制代码其中setsockopt()函数是可选的,一般只用于规避socket()函数的建立错误。 建立了TCP服务器后,返回的socklen_t型实参在后面的HTTP网页服务器中需要用到。 HTTP网页服务器所属的TCP操作是需要另起轮询线程来让客户端进行accept()握手操作的,accept()之前的listen()倒是只需要执行一次即可,accept()握手操作和recv()接收操作需要创建一个死循环线程: - pthread_create(&tid_tcp_web_recv , NULL , Thread_TCP_Web_Recv , NULL);
( t( E Z, ^) h0 }1 f5 P& S - void * Thread_TCP_Web_Recv(void *arg)
" {! t' Y5 M5 h9 ?; C* E - {
$ d. G: M2 L# G4 t - 。。。% X5 w7 C5 c& y# q3 E' W, X- I: i
- while(1)
, Y- B& P7 K4 b( u% I) {/ f) z - {
! E: ~8 A* n) L - fd_socket_conn = accept(socket_web_server , (struct sockaddr *)&sockaddr_in_conn , &addrsize);
" k( Y2 F/ r/ v' v9 H. I2 X# B0 i& A - printf("fd_socket_conn = accept()\n");
4 g4 s& J# [8 P; O4 q N - 。。。
- w! n( x( K( x5 d - recv(fd_socket_conn , recvbuf , 1000 , 0);
* F- s2 x$ X2 \; H$ P# @ - }, w; n1 D+ i$ |) W) |* @6 l1 F# S
- 。。。8 Q" x) z9 z) _' e/ q( E
- }
复制代码MJPG帧可以使用Grab操作获取,获取到的MJPG帧需要在TCP线程中读,在Grab操作线程中写,这种被多个线程访问的资源需要加锁防止读写冲突,即资源被Grab操作写入时,需要上锁,不允许其它线程访问,操作完成时需要解锁,允许其它线程访问: - pthread_mutex_lock(&pmt);* o& [' X. w4 R: q+ S7 C. t5 w0 k/ \
- pic_tmpbuffer = pic.tmpbuffer;
8 T$ z% I8 |. Q r3 I" l2 S - pic.tmpbytesused = buff.bytesused;
" N! s8 i' O# m6 S" q. `: ^+ R4 n - pic_tmpbytesused = pic.tmpbytesused;: h2 R+ G0 |) B# h3 ~; a$ O- _+ ~
- pthread_cond_broadcast(&pct);
' q4 {6 c7 R; Q' ~& C9 y - pthread_mutex_unlock(&pmt);
复制代码线程互斥锁使用之前需要初始化: - pthread_mutex_t pmt;: V' w( W4 N' g$ L+ }
- pthread_cond_t pct;1 z" j$ N/ E# A/ q
- int main(int argc, char* argv[])7 @( ]6 u$ S* p. m w
- {8 D; T2 P6 L. B& c7 K& G8 Y; F
- ...3 B; E# n) Y [$ b2 V
- TCP_Server_Found(&socket_web_server , (char*)argv[2] , PORT_TCP);; S- T. E' a2 o7 X D
- pthread_mutex_init(&pmt , NULL);2 A2 ?# m& e" W: H: |$ z: T; e3 B0 |+ |
- pthread_create(&tid_tcp_web_recv , NULL , Thread_TCP_Web_Recv , NULL);$ o$ b. G4 ^1 ?/ q, H) J
- pthread_create(&tid_tcp_web_send , NULL , Thread_TCP_Web_Send , NULL);
* K: K% h$ j: U. \% E1 b7 F9 M _ - ...3 v0 S8 d C3 h1 J+ W
- while(1)
" c' v8 ?7 ]" z2 ] - {: F0 b N- e7 g7 M9 |
- V4l2_Grab_Mjpeg(false , MJPEG_FILE_NAME);7 P' Z3 N# s) e
- ...* n8 E$ s8 C, p0 j$ q
- }1 u; j& K' J) u
- ...
: q' x( _3 R9 o* L9 ^7 \$ N* w - }
复制代码然后是发送的细节,发送图片文件之前,需要先发送HTTP标准头,这个相当于给发送图片或者其它类型的流数据铺路: - <p style="box-sizing: border-box; border: 0px; vertical-align: baseline; line-height: 26px;"><span style="text-indent: 32px;">
; G9 w, e$ D- Q7 ?& G0 B) ] X# d# f - </span></p><p style="box-sizing: border-box; border: 0px; vertical-align: baseline; line-height: 26px;"></p>
复制代码- #define STD_HEADER "Connection: close\r\n" \) _9 A: _( B( i0 x7 |3 S9 k5 ^+ n1 _5 q
- "Server: MJPG-Streamer/0.2\r\n" \: v7 H) t4 }: j+ B J
- "Cache-Control: no-store, no-cache, must-revalidate, pre-check=0, post-check=0, max-age=0\r\n" \
( \1 w7 f1 S- y8 O - "Pragma: no-cache\r\n" \
, d: d& F9 C( T - "Expires: Mon, 3 Jan 2000 12:34:56 GMT\r\n"" x) A, K- U/ Z& r! s
- #define BOUNDARY "boundarydonotcross"
3 s- J& P/ n( I( ~( s - printf("preparing header\n");
+ M5 h7 ~/ a% P8 K6 l - sprintf(buffer, "HTTP/1.0 200 OK\r\n" \
/ Q& u. k- X |/ J) ~4 W - "Access-Control-Allow-Origin: *\r\n" \& z' J- T( a4 X; c) t
- STD_HEADER \& H0 @& o, n5 L6 c4 {2 V7 _; p
- "Content-Type: multipart/x-mixed-replace;boundary=" BOUNDARY "\r\n" \7 D0 N; n, d# K0 }2 d! ]
- "\r\n" \3 _. q* _: M7 P7 o9 e9 B
- "--" BOUNDARY "\r\n");+ h: D- G! l+ a2 r% r7 J7 g8 Y
- if(write(fd, buffer, strlen(buffer)) < 0)
~( [; T: B8 v. k2 K - {$ U3 `2 E' j4 \; Y: c
- free(frame);' Y" v% ]( J+ j% F) \
- return;- k. l7 y" t2 _9 D; m
- }
复制代码发送完HTTP标准头之后,就需要发送内容头(Content-Type),这处的Content-Type为image/jpeg,同样,HTTP标准协议里面image支持的类型远不止jpeg一种,发送完内容头之后就是正文和boundary结尾,这样帧完整的HTTP头发送到指定的TCP GET地址,就会在浏览器中显示刚刚发送的图片: - <pre class="prettyprint lang-cpp" style="box-sizing: border-box; font-family: Monaco, Menlo, Consolas, "Courier New", monospace; font-size: 16px; white-space: pre-wrap; line-height: 1.38462; color: rgb(51, 51, 51); word-break: break-all; background-color: rgb(245, 245, 245); border: 0px; border-radius: 4px; vertical-align: baseline;"> sprintf(buffer, "Content-Type: image/jpeg\r\n" \* Y' l" Z- \: |2 p$ G
- "Content-Length: %d\r\n" \! [/ K! }# j( `/ H% p+ o
- "X-Timestamp: %d.%06d\r\n" \7 ^4 y& `6 t' V: k6 M
- "\r\n", frame_size, (int)timestamp.tv_sec, (int)timestamp.tv_usec);
) t$ W. D! W5 m5 T3 J+ E% A - printf("sending intemdiate header\n");
. }+ D4 b2 Z J5 |) F: x3 u1 a - if(write(fd, buffer, strlen(buffer)) < 0): ]& Y8 o; f6 S) [/ l w; f
- break;
( v$ c K& z, ]. y* p% q - printf("sending frame\n");
1 e) q' c' J8 d5 ~8 x! p3 F - if(write(fd, frame, frame_size) < 0)2 `4 {4 \, ]8 J0 N% c: Y* P; ]" D/ @! }
- break;
9 `% I6 m, x, W5 d% c: b - printf("sending boundary\n");9 {* o$ k! h& N, ~6 i: p5 T, Y
- sprintf(buffer, "\r\n--" BOUNDARY "\r\n");
$ B# l0 ?* O3 f+ h9 b0 ]8 A - if(write(fd, buffer, strlen(buffer)) < 0)* Y/ h5 l9 j& G) N- `8 ^# r
- break;</pre><p style="box-sizing: border-box; border: 0px; font-size: 16px; vertical-align: baseline; line-height: 26px; color: rgb(51, 51, 51); font-family: " helvetica="" neue",="" helvetica,="" tahoma,="" arial,="" "microsoft="" yahei",="" "hiragino="" sans="" gb",="" "wenquanyi="" micro="" hei",="" sans-serif;"=""></p>
复制代码另外需要说明的是,TCP服务器线程在发送MJPEG流的时候是死循环发送的,因此TCP客户端在发送完GET指令之后,就会收到TCP服务器循环发送的图像缓存,TCP客户端会陷入忙等待状态无法再对外发送任何GET或者POST指令,从客户端使用者角度来看的效果就是网页一直在等待。 
# E) }& I+ D/ R3 T
. L9 x" b9 k" S$ k" y二、UDP上位机UDP发送操作,同样需要先建立UDP Socket:0 W. D/ d8 l9 J' P
- int UDP_Send_Found(socklen_t* socket_found , struct sockaddr_in *addr , char* ip , int port)
9 }* A' k/ l3 R - {
( U$ z- E: W4 _8 E9 @5 p* \ - *socket_found = socket(AF_INET, SOCK_DGRAM, 0);3 B: ?( N* O/ [/ f6 k9 w
- if(*socket_found == (~0))
8 L# ]/ f/ ]" u) D/ D- x! q) L - {
: }6 [. |2 \% [ l' S0 ]' Z - printf("Create udp send socket failed!\n");! c8 W& ~/ n& s# K0 r
- return -1;$ b) }! ?! p( D1 f
- }
( `6 p3 T& T; v( e( `! R# J - addr->sin_family = AF_INET;/ R. e" y/ f0 { K8 S& A
- addr->sin_addr.s_addr = inet_addr(ip);
) U6 f7 c6 g' z( e1 h U - addr->sin_port = htons(port);. R( O+ P5 }; O* O
- memset(addr->sin_zero, 0, 8);
7 G; j2 w3 v6 N/ E4 ^$ d - return 0;+ {& B: A" e5 v& ?0 ?1 J5 t7 r
- }
复制代码
9 D% W5 ~' H' u$ w/ w
5 [ ~; Z& C8 I+ J+ @6 p* y0 r9 m而UDP文件发送则要比HTTP发送简单得多,只需要将文件切片,每一片为固定长度的UDP帧长度,逐帧发送即可:/ v3 i6 L/ |/ U( v
* c+ i$ G$ ?8 [: ~- k& K0 a
) ~% \- a( h% W- O9 Z
- while(fend > 0)
8 i1 B& p( @; Z2 }0 B" A - {( X9 v, g, M/ ~* m; A
- memset(picture.data , 0 , sizeof(picture.data));
$ \$ O: t6 L6 R: r7 j - fread(picture.data , UDP_FRAME_LEN , 1, fp);+ k9 B c9 y7 D" `3 Y4 m. V
- if(fend >= UDP_FRAME_LEN)
6 c8 J6 Y* |- C% q7 X - {/ ?; h; u/ [, w0 V4 O
- picture.length = UDP_FRAME_LEN;
. G& e2 \1 Q9 t. |- G& [/ T* R& L- p, F - picture.fin = 0;
% c+ j8 R5 K2 \8 O6 X4 Q6 p- _9 f. c - }( m: t. H! L* K2 @. P. Z& P
- else
% `* ~" \, ]( X( ~* x - {0 G6 O3 T4 s9 s. H& u2 E' |' b8 {2 k- Y) R
- picture.length = fend;2 ]6 ~5 |" g5 F) m2 _7 l4 P
- picture.fin = 1;! C! j E5 M8 e( Q# R. _. q4 d' p
- }
5 Y2 ~$ o* o: e3 Y) q - //printf("sendbytes = %d \n",sendbytes);3 H8 Z! I7 g; U
- sendbytes = sendto(socket_send, (char *)&picture, sizeof(struct Package), 0, (struct sockaddr*)&addr,addr_len);
6 ` w( f' n3 k! p+ b# H - if(sendbytes == -1)% s7 G* |3 R) |
- {
4 C5 w) b4 V( G7 r1 t' R: U% M - printf("Send Picture Failed!d\n");; h3 a& A3 I# J& [; h
- return -1;* p2 M9 V* C. [+ ?4 U6 T7 j# ?- b
- }1 ~, a" F3 X3 t: l n/ i' |
- else2 I" C. O8 }5 p% B; l3 K
- {. q" r# P; r; ^
- fend -= UDP_FRAME_LEN;$ @8 }) i. n- x) b1 v: v
- }% a7 H4 `! `5 I9 A( t& C
- }
复制代码 7 X: ^3 O. q0 O# s& v0 q
8 V2 t8 z# b) V6 d4 ?, ~/ k6 R ! y, E' y; |, [7 g, e3 u
# T+ ?5 U% g2 N2 u5 tiMX8MPlus 核心板: https://www.forlinx.com/product/136.html |