本帖最后由 飞凌-marketing 于 2022-1-27 11:00 编辑 ! R, m* r- u/ ? ~1 ~3 M1 _
; X# x* o2 H( J! R8 d5 y
2 U" S) l# P+ R, F% M作者|donatello1996 来源 | 电子发烧友 题图|飞凌嵌入式 iMX8MPlus 核心板: https://www.forlinx.com/product/136.html
3 p" Y# M9 C6 }% z1 A3 P; B& Q! |3 ^0 W" s2 I
本文采用的硬件板卡为飞凌嵌入式OKMX8MP-C开发板,系统版本Linux5.4.70+Qt5.15.0,主要介绍基于HTTP网页服务器和UDP上位机的MJPG码流传输。 MJPG格式作为一种持续传输的视频码流,在远程监控领域中应用较广,而实现这种远程监控的第三方应用最常见的有两种:浏览器HTTP网页、UDP上位机。
% A+ |# h! Y/ A- O0 n7 T
, K' K) U* ^8 O5 M2 _: j两者各有优势,对比鲜明,其中: 这两种应用各有优缺点,对于嵌入式开发者来说,两者都必须掌握。
* x+ ^9 e, |3 D, N* E$ {3 \一、HTTP网页服务器1 {$ N {8 m* a& x$ z9 d- m; ]6 h
先说下HTTP网页服务器获取MJPG码流的代码,首先是OKMX8MP-C在开发板端建立TCP服务器: - int TCP_Server_Found(socklen_t* socket_found , char* ip , int port)8 b! \4 X" |: H H- A
- {! f' L2 ]5 B( b. W& m$ ]% F
- struct sockaddr_in servaddr;
% A" @3 _% v% [$ _* k/ D: U - socklen_t addrsize = sizeof(struct sockaddr);- l9 |; U! I+ }& X( e( r# G/ E$ x
- bzero(&servaddr , sizeof(servaddr));* o4 E* U/ X9 d# S4 Q6 E6 |; y* ~
- servaddr.sin_family = AF_INET;
0 Z! `' u5 F6 }$ ~# t' ~4 g' ` - servaddr.sin_addr.s_addr = inet_addr(ip);5 O6 X, a* B/ f; x
- servaddr.sin_port = htons(port);0 N" @0 w) W5 |5 `6 Q. G. E; f
- int ret;
d9 q7 T' X9 U0 F+ _ - IF( (*socket_found = socket(AF_INET , SOCK_STREAM , 0)) == -1); W9 I! N: `/ F- y$ R7 d- m
- {# p" z3 S3 b9 w7 }. \5 H. I
- printf("Create socket error: %s (errno :%d)\n",strerror(errno),errno);! ~* ~9 _& E- @, m0 B
- return -1;
9 ]( g9 f2 y: y/ m, n J' E9 [8 S; A - }' M$ ]: T; j' }
- int on = 1;
& \, V. q) M& M) c7 i* w: } - if(setsockopt(*socket_found , SOL_SOCKET, SO_REUSEADDR, &on, sizeof(on)) < 0)
7 u; a* o. U/ | - {$ d+ K8 J" e$ F7 v( u, U3 w* o/ l3 x
- printf("setsockopt error\n");3 t; J" {0 f( z2 C) {- F
- }% }- \ j' X( `3 B
- ret = bind(*socket_found , (struct sockaddr *)&servaddr , addrsize);! m& G& s* a) w2 G! z6 O8 n& c& L5 z
- if(ret == -1)1 {6 l" D7 T& j; Z
- {# m5 l/ z& Z4 F( c( u& l# S: d
- printf("Tcp bind faiLED!\n");3 N7 Y! R* I' @; d' v
- return -1;; \( O/ o+ K. w! K/ k5 L
- }
" ?% |8 u* d7 k8 E7 b; O - if(listen(*socket_found , 5) == -1)0 S( O' w$ z+ E# g
- {* E( M2 s3 [8 {% K; L' Z# V
- printf("Listen failed!\n");
: Q+ m. C3 z5 c. ?6 } - return -1;/ O/ s8 t0 h! b7 L6 J
- }
0 O% N. W) g o/ X9 g3 W - return 0;, h# |1 A$ Q" A# w' K# M0 B. b8 ^7 f
- }
复制代码其中setsockopt()函数是可选的,一般只用于规避socket()函数的建立错误。 建立了TCP服务器后,返回的socklen_t型实参在后面的HTTP网页服务器中需要用到。 HTTP网页服务器所属的TCP操作是需要另起轮询线程来让客户端进行accept()握手操作的,accept()之前的listen()倒是只需要执行一次即可,accept()握手操作和recv()接收操作需要创建一个死循环线程: - pthread_create(&tid_tcp_web_recv , NULL , Thread_TCP_Web_Recv , NULL);
1 t3 L5 t3 L8 c, e- [" J1 ~. D b/ E - void * Thread_TCP_Web_Recv(void *arg)
- _3 @, v7 r/ ?$ Z; m* o, r - {& F: ]; P0 R% e$ M
- 。。。4 e" M, B% M( O7 t/ F* }
- while(1)1 A+ g1 e: Q3 `
- {7 X) m ]/ A+ [) X6 \5 Y- X
- fd_socket_conn = accept(socket_web_server , (struct sockaddr *)&sockaddr_in_conn , &addrsize);
$ P; N5 u8 \ N% M* q - printf("fd_socket_conn = accept()\n");* i) P7 C7 A1 P9 F/ R/ S
- 。。。
5 d% C) f& q/ a/ o5 H - recv(fd_socket_conn , recvbuf , 1000 , 0);5 o2 X/ f/ j1 ?" O2 D. d% Z
- }6 y5 B0 E) f+ [7 ^
- 。。。
; ~2 r Y# | z5 B) I, R - }
复制代码MJPG帧可以使用Grab操作获取,获取到的MJPG帧需要在TCP线程中读,在Grab操作线程中写,这种被多个线程访问的资源需要加锁防止读写冲突,即资源被Grab操作写入时,需要上锁,不允许其它线程访问,操作完成时需要解锁,允许其它线程访问: - pthread_mutex_lock(&pmt);
: l: ?% e; e( }* @ - pic_tmpbuffer = pic.tmpbuffer;4 J' l! y0 N: L4 K* N, a6 g
- pic.tmpbytesused = buff.bytesused;
% y% ]3 s) q) o; W. @ - pic_tmpbytesused = pic.tmpbytesused;
! W9 G, C3 T9 O - pthread_cond_broadcast(&pct);
, v2 u" l0 v" P: l" C# g: f" f# F9 j! b1 A - pthread_mutex_unlock(&pmt);
复制代码线程互斥锁使用之前需要初始化: - pthread_mutex_t pmt;$ B/ _8 i, U" P4 d9 R
- pthread_cond_t pct;
# g# p& j* E7 a9 E5 X5 m0 _0 y R4 k: v - int main(int argc, char* argv[])
0 O" }6 J+ L% \% @7 N; \ - {
# E( l; Z# k: Q) B+ X5 @ - ...
' X* T+ i3 J8 U0 j$ {1 h7 T$ U - TCP_Server_Found(&socket_web_server , (char*)argv[2] , PORT_TCP);
* f$ P: X( Z; c( o& x* |+ J - pthread_mutex_init(&pmt , NULL);/ D- z& T# r0 B5 d0 Y, S
- pthread_create(&tid_tcp_web_recv , NULL , Thread_TCP_Web_Recv , NULL);
! ~# A6 Q" E3 Q$ O5 @ - pthread_create(&tid_tcp_web_send , NULL , Thread_TCP_Web_Send , NULL);
p1 s' G" c3 }; ~; r - .../ q* Q& V( b9 X' ~
- while(1)
! [7 O: P- B. i - {
5 u+ I. D; n' t* e0 R - V4l2_Grab_Mjpeg(false , MJPEG_FILE_NAME);4 @4 C/ J3 p" ~! Y0 Q8 Z
- .../ X+ j3 C+ s! b: ]
- }
3 F% l* A A* H \) U( h - ...; G6 [" k3 g( d+ ?$ s; W
- }
复制代码然后是发送的细节,发送图片文件之前,需要先发送HTTP标准头,这个相当于给发送图片或者其它类型的流数据铺路: - <p style="box-sizing: border-box; border: 0px; vertical-align: baseline; line-height: 26px;"><span style="text-indent: 32px;">
% D/ i) j/ E$ k ] - </span></p><p style="box-sizing: border-box; border: 0px; vertical-align: baseline; line-height: 26px;"></p>
复制代码- #define STD_HEADER "Connection: close\r\n" \
* K* S% H* Q$ P' C: |+ L; l - "Server: MJPG-Streamer/0.2\r\n" \4 ?& i7 d# S& Q8 W5 {; o: J2 }: W
- "Cache-Control: no-store, no-cache, must-revalidate, pre-check=0, post-check=0, max-age=0\r\n" \0 w7 N; H, A1 t+ {
- "Pragma: no-cache\r\n" \
" i" S4 v* b; ^8 C - "Expires: Mon, 3 Jan 2000 12:34:56 GMT\r\n"
( X- b( K2 J4 E _0 o0 p - #define BOUNDARY "boundarydonotcross") D/ k4 v. _+ } A; l( I* t
- printf("preparing header\n");
" P% ~! f4 }" u - sprintf(buffer, "HTTP/1.0 200 OK\r\n" \& f1 b' S# ]4 S3 d8 `
- "Access-Control-Allow-Origin: *\r\n" \
( c5 b/ s- D F5 O2 I: D3 f) o - STD_HEADER \* f! I; F& T+ z8 z
- "Content-Type: multipart/x-mixed-replace;boundary=" BOUNDARY "\r\n" \
( }+ K: _" V m+ E R9 ?' g - "\r\n" \
# H! b, N. T' t, j |% ]0 G+ X; ^, r - "--" BOUNDARY "\r\n");
- F6 ~4 `/ i2 F) Y* n# @5 N - if(write(fd, buffer, strlen(buffer)) < 0), k" S' K* \3 L2 I! w* T& v. P
- {# E3 k2 m4 g& Z: y5 Y* S, G" e
- free(frame);/ p. d( f- K4 T% v
- return;) {: e1 @5 G% ]4 t1 t* U3 F% N
- }
复制代码发送完HTTP标准头之后,就需要发送内容头(Content-Type),这处的Content-Type为image/jpeg,同样,HTTP标准协议里面image支持的类型远不止jpeg一种,发送完内容头之后就是正文和boundary结尾,这样帧完整的HTTP头发送到指定的TCP GET地址,就会在浏览器中显示刚刚发送的图片: - <pre class="prettyprint lang-cpp" style="box-sizing: border-box; font-family: Monaco, Menlo, Consolas, "Courier New", monospace; font-size: 16px; white-space: pre-wrap; line-height: 1.38462; color: rgb(51, 51, 51); word-break: break-all; background-color: rgb(245, 245, 245); border: 0px; border-radius: 4px; vertical-align: baseline;"> sprintf(buffer, "Content-Type: image/jpeg\r\n" \
/ i% c' R& p+ D/ M - "Content-Length: %d\r\n" \
6 |) C& C8 y8 a9 m; g - "X-Timestamp: %d.%06d\r\n" \
: l' {& J: K8 X7 T/ I - "\r\n", frame_size, (int)timestamp.tv_sec, (int)timestamp.tv_usec);
1 G* B3 M3 _' S) F - printf("sending intemdiate header\n");
* z2 `0 V% Z5 M( \. F - if(write(fd, buffer, strlen(buffer)) < 0)
) ~* a- ]4 J+ e& @# q; v4 S - break;
- c" S% _6 F. @5 G8 U2 V - printf("sending frame\n");& Y# b* |! B3 m, e0 Y) b, ?
- if(write(fd, frame, frame_size) < 0); ^' f9 ?! T/ P9 l- ^! z
- break;
- E/ [. Y- ?- _- s7 Y7 } - printf("sending boundary\n");% L) f8 ^: }! R! O( D
- sprintf(buffer, "\r\n--" BOUNDARY "\r\n");
4 _2 g a$ f; i/ l w/ B# |' b - if(write(fd, buffer, strlen(buffer)) < 0)4 I" ^2 F7 C u: u( C9 B& M
- break;</pre><p style="box-sizing: border-box; border: 0px; font-size: 16px; vertical-align: baseline; line-height: 26px; color: rgb(51, 51, 51); font-family: " helvetica="" neue",="" helvetica,="" tahoma,="" arial,="" "microsoft="" yahei",="" "hiragino="" sans="" gb",="" "wenquanyi="" micro="" hei",="" sans-serif;"=""></p>
复制代码另外需要说明的是,TCP服务器线程在发送MJPEG流的时候是死循环发送的,因此TCP客户端在发送完GET指令之后,就会收到TCP服务器循环发送的图像缓存,TCP客户端会陷入忙等待状态无法再对外发送任何GET或者POST指令,从客户端使用者角度来看的效果就是网页一直在等待。  ( ?0 Q$ L+ \# l. |
 # R6 S- u3 L7 Q6 K
二、UDP上位机UDP发送操作,同样需要先建立UDP Socket:: S) {0 O# B5 j' j+ N3 j
- int UDP_Send_Found(socklen_t* socket_found , struct sockaddr_in *addr , char* ip , int port), A6 e+ k7 H: E, l
- {" Q. l; `- n) k) i7 v- @; n/ F
- *socket_found = socket(AF_INET, SOCK_DGRAM, 0);
D' p# {7 u3 b, z! {0 e8 u - if(*socket_found == (~0))- n% |* y% K: W- n0 _
- {4 E" J0 t8 K# ]+ |
- printf("Create udp send socket failed!\n");: n) r) P2 c D3 |+ n% l, v1 o
- return -1;) }; [. R: A8 B- ~; a
- }3 k' ~' [1 b7 O; p# P$ J/ i7 `
- addr->sin_family = AF_INET;
5 m" c$ W5 P; M - addr->sin_addr.s_addr = inet_addr(ip);
: p6 J1 m Q% E3 N8 F - addr->sin_port = htons(port);) |8 |2 m/ K& W ~
- memset(addr->sin_zero, 0, 8);
" z" W$ E( @' F- m - return 0;5 s1 }, I; ^- h3 y
- }
复制代码
, d( W1 U" f4 V& h; u. l. ?
* a" m2 y6 C0 h% r/ n9 { c而UDP文件发送则要比HTTP发送简单得多,只需要将文件切片,每一片为固定长度的UDP帧长度,逐帧发送即可:* _& w- G ?( ^8 N. _; K( [! s
5 ?% A- ~% K5 s$ V3 J: n4 U T
+ d: T% p, x9 R* Q! q+ M. r- `0 A- while(fend > 0)2 {# M; O1 T! ?2 |& J6 Y& M$ W
- {
( b2 x8 l" k+ e! U8 x% m9 {4 K* s - memset(picture.data , 0 , sizeof(picture.data));
, g+ O/ v* n( a/ G3 [ - fread(picture.data , UDP_FRAME_LEN , 1, fp);
5 Z4 T! v* [* B - if(fend >= UDP_FRAME_LEN)
9 h3 J3 B5 i: G" p2 Z1 k - {
; ?! i3 y- _( ?4 q, ? - picture.length = UDP_FRAME_LEN;
5 H# g6 y' s8 b8 x$ h5 C3 w7 r. T - picture.fin = 0; n6 m# Q* Y ?# { i
- }+ Z6 A& X; {, G% v% `
- else
" ?' c% ~ f7 |2 q$ c# T, i% O, R% | - {
* m$ E8 r- L( L7 H" K4 \ - picture.length = fend;
6 b* [1 I ^' M" p; X. p - picture.fin = 1;) g0 U5 T4 e; o, a; D, L
- }
2 z! Q/ J2 I* ]$ k r7 t( {+ Z5 d1 ^& Z - //printf("sendbytes = %d \n",sendbytes);
: {5 Z4 v4 k: M8 ~" d - sendbytes = sendto(socket_send, (char *)&picture, sizeof(struct Package), 0, (struct sockaddr*)&addr,addr_len);
: H1 T% P5 U1 Q, a - if(sendbytes == -1)9 D: c1 p' h; C4 d5 Z& Q+ v
- {3 }6 t+ r* D. E9 J: K# s8 _
- printf("Send Picture Failed!d\n");: D! C2 `+ a! z' I/ H
- return -1;+ [ E& W5 `; V1 e. J' X4 O
- }+ z' H) S* x+ V7 V/ a* Q. m
- else3 j) Q, M9 ~# M }
- {7 ^' D* ^2 l; c! {' I0 @
- fend -= UDP_FRAME_LEN;
! k" Z* t. e6 N: c0 R$ W/ [ - }- q, i0 Y1 w3 g1 [
- }
复制代码 " i; `, M3 u$ q k
$ G; h8 v k# j5 B$ g. A D, u) ]# u
 * T4 x0 l$ o' s4 ?' H
1 B/ B- [. Y" f5 }/ n9 NiMX8MPlus 核心板: https://www.forlinx.com/product/136.html |