本帖最后由 飞凌-marketing 于 2022-1-27 11:00 编辑
" W9 N. l6 L4 U! c2 R
5 t+ W7 m$ |+ a% P
: e5 b( H7 M+ N4 Y# E作者|donatello1996 来源 | 电子发烧友 题图|飞凌嵌入式 iMX8MPlus 核心板: https://www.forlinx.com/product/136.html. z7 z8 X+ Q0 y: t- x: @5 e7 X
: L" ^, {- G; r
本文采用的硬件板卡为飞凌嵌入式OKMX8MP-C开发板,系统版本Linux5.4.70+Qt5.15.0,主要介绍基于HTTP网页服务器和UDP上位机的MJPG码流传输。 MJPG格式作为一种持续传输的视频码流,在远程监控领域中应用较广,而实现这种远程监控的第三方应用最常见的有两种:浏览器HTTP网页、UDP上位机。
4 ^6 _" `) A+ u- d" [, o; b ( o9 I$ b7 C/ i. s. }0 i
两者各有优势,对比鲜明,其中: 这两种应用各有优缺点,对于嵌入式开发者来说,两者都必须掌握。 * @4 ~1 z' ?4 c3 F6 F+ q+ r
一、HTTP网页服务器
; r8 Y7 m* C* z8 _: Q先说下HTTP网页服务器获取MJPG码流的代码,首先是OKMX8MP-C在开发板端建立TCP服务器: - int TCP_Server_Found(socklen_t* socket_found , char* ip , int port)' }" z6 g, g: H. i: e9 i: U" y
- {) ]. |6 J2 @( V* b2 W
- struct sockaddr_in servaddr;( D* `, r) w4 s# H7 s
- socklen_t addrsize = sizeof(struct sockaddr);! \1 H0 L& j( x6 Q7 }+ k" B
- bzero(&servaddr , sizeof(servaddr));2 _- @/ M7 t1 ]% Z
- servaddr.sin_family = AF_INET;1 I9 _( c& N% m+ ?0 G" F, ?3 Z
- servaddr.sin_addr.s_addr = inet_addr(ip);9 V+ e8 S, |' y( a, P5 F
- servaddr.sin_port = htons(port);# E6 ^9 a" _, @/ p
- int ret;
% f4 ^* x. G$ Q - IF( (*socket_found = socket(AF_INET , SOCK_STREAM , 0)) == -1)
! F; h* D `6 s' n$ R' D - {
# s; w! O9 ^- i3 W- h - printf("Create socket error: %s (errno :%d)\n",strerror(errno),errno);9 n' j9 Z& v; k
- return -1;, y/ ^! z2 a: Y; h5 i
- }6 j5 h! w4 M5 h- J6 e/ y& K/ y
- int on = 1;3 b3 \# z# s* S/ {5 V# t
- if(setsockopt(*socket_found , SOL_SOCKET, SO_REUSEADDR, &on, sizeof(on)) < 0)
- u3 p. G, S( d1 N0 |1 F - {
# z6 R$ F( a) Y# i! W( v - printf("setsockopt error\n");: V7 e, [- V0 N6 b6 U
- }
1 ?: q5 s- w2 E' i - ret = bind(*socket_found , (struct sockaddr *)&servaddr , addrsize);& e6 r3 c; s, I$ X
- if(ret == -1)
/ P+ l; O; r3 ^+ `7 N' C - {
. \, e" ^. Q' U2 E - printf("Tcp bind faiLED!\n");
: K# v g0 b: y - return -1;/ C. a: q! c- }& \
- }
9 c, o2 I% U% v/ D2 b% _; |, M - if(listen(*socket_found , 5) == -1)
4 {8 R% K/ {* {& r - {
9 k, {+ S* U2 |$ U. I6 o - printf("Listen failed!\n");
! O1 B. e& K8 R! z7 o - return -1;
! k) U: N# k) u2 K0 b - }
1 b& a0 X! R" [# E$ `+ { - return 0;
' E5 R/ D0 o, H& m! M4 s6 ^ - }
复制代码其中setsockopt()函数是可选的,一般只用于规避socket()函数的建立错误。 建立了TCP服务器后,返回的socklen_t型实参在后面的HTTP网页服务器中需要用到。 HTTP网页服务器所属的TCP操作是需要另起轮询线程来让客户端进行accept()握手操作的,accept()之前的listen()倒是只需要执行一次即可,accept()握手操作和recv()接收操作需要创建一个死循环线程: - pthread_create(&tid_tcp_web_recv , NULL , Thread_TCP_Web_Recv , NULL);# V' R* }6 C3 H( L) p" I+ b
- void * Thread_TCP_Web_Recv(void *arg)2 T0 [: A6 E3 N- a, J' W
- {
& R; J9 U. y& [, b% a" h4 k7 C - 。。。0 n( N" V# I# ^1 e
- while(1)0 J7 D, L9 ^3 d! n5 ^" W ^0 M6 ?4 x
- {0 W. C$ h1 u0 Q/ a* Y8 R* L- u
- fd_socket_conn = accept(socket_web_server , (struct sockaddr *)&sockaddr_in_conn , &addrsize);
2 x r J4 h9 B' _8 Q - printf("fd_socket_conn = accept()\n");0 b% |! R9 H' Z, L1 ]
- 。。。. G7 L! _/ J- c3 {& q5 `- M! _; g
- recv(fd_socket_conn , recvbuf , 1000 , 0);0 { ]4 l) A# ]9 ? Q. y2 D
- }3 U4 N, W# U0 K$ h* G: j
- 。。。
9 r4 S' W; D$ A# K' L2 K! ? - }
复制代码MJPG帧可以使用Grab操作获取,获取到的MJPG帧需要在TCP线程中读,在Grab操作线程中写,这种被多个线程访问的资源需要加锁防止读写冲突,即资源被Grab操作写入时,需要上锁,不允许其它线程访问,操作完成时需要解锁,允许其它线程访问: - pthread_mutex_lock(&pmt);4 L. M& |4 k B3 u7 W; T: d- L
- pic_tmpbuffer = pic.tmpbuffer;
1 x, i3 R2 o1 L" l/ E5 S! L. Z* x6 Y - pic.tmpbytesused = buff.bytesused;
: T$ v7 {5 m) I - pic_tmpbytesused = pic.tmpbytesused;$ v }( z$ \0 d q0 O: @: _
- pthread_cond_broadcast(&pct);
" g) h$ x' l" F* t - pthread_mutex_unlock(&pmt);
复制代码线程互斥锁使用之前需要初始化: - pthread_mutex_t pmt;
A& Q0 q2 J6 h2 h - pthread_cond_t pct;
2 p- O& v& T% L# }' Z - int main(int argc, char* argv[])
% f# }1 N+ b+ L/ n' v. ~0 j - {
; g2 G* q3 j4 K8 G5 a - ...& L( u% c/ I* O+ Q: @( t
- TCP_Server_Found(&socket_web_server , (char*)argv[2] , PORT_TCP);) {7 r6 q" G9 j B9 w4 l
- pthread_mutex_init(&pmt , NULL);: i$ T6 V8 \% A7 ?- l
- pthread_create(&tid_tcp_web_recv , NULL , Thread_TCP_Web_Recv , NULL);' n/ l. f0 A' w; ~* q
- pthread_create(&tid_tcp_web_send , NULL , Thread_TCP_Web_Send , NULL);# r- B7 g, r/ c5 w3 w( P
- ...8 D( `0 v, W; S
- while(1)
& l" Y$ J0 M9 d5 b - {) h/ a3 C D5 F! k/ P
- V4l2_Grab_Mjpeg(false , MJPEG_FILE_NAME);% O( s6 w1 q& `3 Y2 u$ ~/ u$ Q3 q
- ..." d! @( p* @) H6 O, V9 t. I8 x3 A: h
- }* _+ S# T, r: C+ Q9 @9 k- [
- ...: T* w2 N' j$ u* @
- }
复制代码然后是发送的细节,发送图片文件之前,需要先发送HTTP标准头,这个相当于给发送图片或者其它类型的流数据铺路: - <p style="box-sizing: border-box; border: 0px; vertical-align: baseline; line-height: 26px;"><span style="text-indent: 32px;">
& a5 }* J: G0 J/ ^6 a! A- p - </span></p><p style="box-sizing: border-box; border: 0px; vertical-align: baseline; line-height: 26px;"></p>
复制代码- #define STD_HEADER "Connection: close\r\n" \$ v! l, l+ J8 s3 l
- "Server: MJPG-Streamer/0.2\r\n" \
: _8 K6 T9 E$ _8 m% m" z0 @ - "Cache-Control: no-store, no-cache, must-revalidate, pre-check=0, post-check=0, max-age=0\r\n" \6 O+ a8 |; U( g0 J$ [
- "Pragma: no-cache\r\n" \
0 E: Z! g1 v6 a% I - "Expires: Mon, 3 Jan 2000 12:34:56 GMT\r\n"& N: R: Z# C9 n7 V
- #define BOUNDARY "boundarydonotcross"
* W) n% y: Z" x6 \4 U0 C% ~5 ~ - printf("preparing header\n");
8 O: X5 y9 Y7 Q - sprintf(buffer, "HTTP/1.0 200 OK\r\n" \$ V# e& e' I& A6 ^4 B+ O8 R
- "Access-Control-Allow-Origin: *\r\n" \$ q& ^0 n: p" Y$ _0 x3 g, s
- STD_HEADER \
, G" G: c/ R" ^7 Z6 F. | - "Content-Type: multipart/x-mixed-replace;boundary=" BOUNDARY "\r\n" \
& B# A# t$ m5 t8 x& k" j - "\r\n" \+ Q8 h$ e6 x7 L, ^
- "--" BOUNDARY "\r\n");
' w9 K Y* P0 K, \4 i, R! Z - if(write(fd, buffer, strlen(buffer)) < 0)9 I1 T( Q4 U- m7 T Y3 @, N0 v6 f" t
- {/ n) ^% r+ ^9 T5 q
- free(frame);* L- i- c3 C a- {+ o; R# P
- return;8 ?9 B u8 e6 f& O9 f
- }
复制代码发送完HTTP标准头之后,就需要发送内容头(Content-Type),这处的Content-Type为image/jpeg,同样,HTTP标准协议里面image支持的类型远不止jpeg一种,发送完内容头之后就是正文和boundary结尾,这样帧完整的HTTP头发送到指定的TCP GET地址,就会在浏览器中显示刚刚发送的图片: - <pre class="prettyprint lang-cpp" style="box-sizing: border-box; font-family: Monaco, Menlo, Consolas, "Courier New", monospace; font-size: 16px; white-space: pre-wrap; line-height: 1.38462; color: rgb(51, 51, 51); word-break: break-all; background-color: rgb(245, 245, 245); border: 0px; border-radius: 4px; vertical-align: baseline;"> sprintf(buffer, "Content-Type: image/jpeg\r\n" \
, E- E: ^& p3 o+ j: j/ v0 V2 B7 ? - "Content-Length: %d\r\n" \) j0 b4 J: V) \: T' g0 ^, F
- "X-Timestamp: %d.%06d\r\n" \7 v3 T) q2 X! S9 i& |
- "\r\n", frame_size, (int)timestamp.tv_sec, (int)timestamp.tv_usec);
$ k* g7 ]$ `* M' X - printf("sending intemdiate header\n");
! a- \ q* f! I; C5 t# J% `) D8 ? - if(write(fd, buffer, strlen(buffer)) < 0), C" _4 u' O- Y3 W4 |- O/ `# v
- break;
3 j- {% S1 C1 u5 o' N - printf("sending frame\n");
2 `; }8 U" W! {$ E0 d, F' Q3 I - if(write(fd, frame, frame_size) < 0)
8 q( N& t( }+ B- U; w - break;
3 Q9 \' I. G/ _3 I* {: [0 N0 m6 W - printf("sending boundary\n");
. e' P: {9 o% a - sprintf(buffer, "\r\n--" BOUNDARY "\r\n");
+ {- E: y( u1 M8 m - if(write(fd, buffer, strlen(buffer)) < 0)4 |4 }8 C3 A$ k% e, _
- break;</pre><p style="box-sizing: border-box; border: 0px; font-size: 16px; vertical-align: baseline; line-height: 26px; color: rgb(51, 51, 51); font-family: " helvetica="" neue",="" helvetica,="" tahoma,="" arial,="" "microsoft="" yahei",="" "hiragino="" sans="" gb",="" "wenquanyi="" micro="" hei",="" sans-serif;"=""></p>
复制代码另外需要说明的是,TCP服务器线程在发送MJPEG流的时候是死循环发送的,因此TCP客户端在发送完GET指令之后,就会收到TCP服务器循环发送的图像缓存,TCP客户端会陷入忙等待状态无法再对外发送任何GET或者POST指令,从客户端使用者角度来看的效果就是网页一直在等待。  2 e* r/ `8 g& s7 g4 a( n+ v

' g* z" n! ]2 @# [6 X6 \二、UDP上位机UDP发送操作,同样需要先建立UDP Socket:
% J5 d2 H- }* X W/ I6 B- int UDP_Send_Found(socklen_t* socket_found , struct sockaddr_in *addr , char* ip , int port)7 |) b: p. r5 |+ E3 S$ E y
- {
% z/ |2 o5 A; M" z( y# k- m - *socket_found = socket(AF_INET, SOCK_DGRAM, 0);
5 O5 B" U, f" o, G - if(*socket_found == (~0))
1 V# D, U' ]; o2 B( A/ d - {
" f4 @/ P/ K: t# v Y - printf("Create udp send socket failed!\n");" u% f1 P! o" v. P; L% Y
- return -1;
* V* s/ U& _6 V6 B' F9 @ - }% V& @9 j5 X* ^4 a% G5 @5 L
- addr->sin_family = AF_INET;
# _5 m) g: ^- }- k$ }9 ^ - addr->sin_addr.s_addr = inet_addr(ip);
5 g2 i" q) _6 u$ [% O - addr->sin_port = htons(port);8 R9 [8 b% c: b) ^" X5 }: p
- memset(addr->sin_zero, 0, 8);
/ }2 {, D4 ~- V5 N7 c - return 0;
5 b5 P/ r. ~- N2 t+ L/ a9 ~7 l% ^" R - }
复制代码
* m! g7 x- Q& K! ] s8 b! v+ M' w& U( F6 g& `4 @& p+ Y
而UDP文件发送则要比HTTP发送简单得多,只需要将文件切片,每一片为固定长度的UDP帧长度,逐帧发送即可:- D8 w5 d* D& g2 A1 n6 F
' W/ i. ?- {6 V: d
: m8 p( M# X; J& X4 i1 X% o
- while(fend > 0)
4 Z, o K4 k E6 z - {
- h' i+ a' ~ F, }1 R; J# P# e - memset(picture.data , 0 , sizeof(picture.data));7 q2 K3 L5 }* `) M- R
- fread(picture.data , UDP_FRAME_LEN , 1, fp);. g8 l% g) e2 f V. ~
- if(fend >= UDP_FRAME_LEN)
3 I8 G+ q( \0 d: V - {
0 A" f" Q7 {) k7 W9 @1 _- L/ B/ U - picture.length = UDP_FRAME_LEN;: }) p% i/ C. M; W
- picture.fin = 0;
1 S' t2 X" S8 R7 | - }
* E. @7 N- h* Y! { - else* i& S% V/ S# m6 n- t4 f* {: ?
- {
2 r0 ^) E' G; N2 \6 ` i3 L$ s. k; Z - picture.length = fend;
* e$ ?: n9 l4 r) W @( Y - picture.fin = 1;
$ d Y% t3 [' {, i7 p - }
, k5 I8 J1 k; o) N - //printf("sendbytes = %d \n",sendbytes);
/ ?. u* i( ?. c+ {4 A - sendbytes = sendto(socket_send, (char *)&picture, sizeof(struct Package), 0, (struct sockaddr*)&addr,addr_len); w% V9 {" A/ c4 r/ X
- if(sendbytes == -1)
' N7 d" F, d7 B m7 x; ^ - {" Z5 [6 I W% n. i9 Z+ ?
- printf("Send Picture Failed!d\n");6 G- v( W) e/ m* Z7 |
- return -1;* f0 A8 h5 v; n6 J) [, b2 l
- }
( B& {) J4 l, l - else
$ o; H, V* e% R( M R# y6 S1 ? - {
/ Q) w/ c/ @) ~1 X" E) o - fend -= UDP_FRAME_LEN;
+ m; J- A; A0 n0 B" `% o& p - }; u! I2 f B3 \1 S! {
- }
复制代码 6 G( c/ ]2 n$ _/ f" v- M C% G
/ O# Y1 Q& e, d" {
 ! G& a4 f& _; I3 x6 [
0 o9 Z' l- t' }5 F4 p' B
iMX8MPlus 核心板: https://www.forlinx.com/product/136.html |