本帖最后由 飞凌-marketing 于 2022-1-27 11:00 编辑
% \ k" `* s0 G. s1 |9 X3 i2 [& u# c- @0 K" ]$ t/ G

5 x' j- B% F F. o5 a3 k' H" n6 K作者|donatello1996 来源 | 电子发烧友 题图|飞凌嵌入式 iMX8MPlus 核心板: https://www.forlinx.com/product/136.html
- i) H- Z+ ?# Q% @# P5 a1 I- S6 ^
/ d/ M) u1 W; J) N( |1 M0 r% ~本文采用的硬件板卡为飞凌嵌入式OKMX8MP-C开发板,系统版本Linux5.4.70+Qt5.15.0,主要介绍基于HTTP网页服务器和UDP上位机的MJPG码流传输。 MJPG格式作为一种持续传输的视频码流,在远程监控领域中应用较广,而实现这种远程监控的第三方应用最常见的有两种:浏览器HTTP网页、UDP上位机。 1 _& r: c' o5 q9 k; C
 9 G* \5 x5 b2 X! |0 a, T6 Y4 `
两者各有优势,对比鲜明,其中: 这两种应用各有优缺点,对于嵌入式开发者来说,两者都必须掌握。 + Z0 t! z4 Z' O* U, r1 V, ]1 l+ @
一、HTTP网页服务器 l7 d) Y/ [/ u x$ `) ~6 @
先说下HTTP网页服务器获取MJPG码流的代码,首先是OKMX8MP-C在开发板端建立TCP服务器: - int TCP_Server_Found(socklen_t* socket_found , char* ip , int port)
9 q- q) g5 T( A) s. N - {
6 a% R- a+ b/ _5 Q" ? - struct sockaddr_in servaddr;$ n& q V( d9 t. V
- socklen_t addrsize = sizeof(struct sockaddr);; e9 Z. b* r6 N2 y0 C6 y
- bzero(&servaddr , sizeof(servaddr));! K1 o. c J1 g8 {9 @$ U! A
- servaddr.sin_family = AF_INET;0 j( v1 w6 P) h& L
- servaddr.sin_addr.s_addr = inet_addr(ip);" r S; z! R! X l, g
- servaddr.sin_port = htons(port);. E1 i& w# n) e' }8 L# d
- int ret;9 \8 |' F# a& F
- IF( (*socket_found = socket(AF_INET , SOCK_STREAM , 0)) == -1)
8 W# V7 N; k& j% d8 A) Y9 d! ` - {+ a# m m' h2 [1 f* F* z3 i2 q, c
- printf("Create socket error: %s (errno :%d)\n",strerror(errno),errno);
5 {* i% W: A% D) W3 J - return -1;; }& T: s- a' M! O
- }
+ a+ m5 G- @# I) c4 \8 o; P - int on = 1;: W* ~* S8 A5 M0 c& g+ \
- if(setsockopt(*socket_found , SOL_SOCKET, SO_REUSEADDR, &on, sizeof(on)) < 0)# R1 H0 i* c0 G$ N, Z6 J& |
- {/ ^1 P6 u( M' a- D0 S+ }
- printf("setsockopt error\n");# [/ z6 g, P! K8 D& ^
- }
- @0 D a0 B' \' J0 u - ret = bind(*socket_found , (struct sockaddr *)&servaddr , addrsize);
. F* j; S" K0 z) K5 N% N& q% T* T - if(ret == -1)
) A1 m3 b: a) [ - { {" [8 i, D4 O% K( @4 q
- printf("Tcp bind faiLED!\n");
; d' m# T# Q* T0 E3 w: a - return -1;
4 Y+ r- f( A7 M: W$ N: ~ - }# t% w* X, x' O3 B' V6 Y
- if(listen(*socket_found , 5) == -1)
- @6 R/ T! |, K& h% ^ - {5 _" i+ q8 s! k7 L# F3 N
- printf("Listen failed!\n");
, R( |6 j4 N9 F Z7 d3 ^# | - return -1;, i" g( {4 k! K. j
- }0 ?9 \2 M/ \' g' e# q* ?
- return 0;
, y3 {% C" ?; N8 C - }
复制代码其中setsockopt()函数是可选的,一般只用于规避socket()函数的建立错误。 建立了TCP服务器后,返回的socklen_t型实参在后面的HTTP网页服务器中需要用到。 HTTP网页服务器所属的TCP操作是需要另起轮询线程来让客户端进行accept()握手操作的,accept()之前的listen()倒是只需要执行一次即可,accept()握手操作和recv()接收操作需要创建一个死循环线程: - pthread_create(&tid_tcp_web_recv , NULL , Thread_TCP_Web_Recv , NULL);$ K; u3 Q7 N+ }$ t
- void * Thread_TCP_Web_Recv(void *arg)
+ w/ i8 ]5 z2 y# `( v7 ?* h - {
: |, t. d1 f: x1 U5 z. c - 。。。- ~& C; m$ `! O& x% x! Q1 K: I
- while(1)( H7 L E- h: V; S8 y4 c
- {$ Q9 c( |) G! Q5 o
- fd_socket_conn = accept(socket_web_server , (struct sockaddr *)&sockaddr_in_conn , &addrsize);
$ K# @6 E& p/ N) F g) b - printf("fd_socket_conn = accept()\n");
& ?4 b+ D" _6 h+ p - 。。。' D4 j1 W! K ]" a8 A7 V% Q+ s
- recv(fd_socket_conn , recvbuf , 1000 , 0);
) M% A- Y' s, g - }& M# b* s$ |5 i4 @1 G3 M
- 。。。
' b2 @* B( y# q - }
复制代码MJPG帧可以使用Grab操作获取,获取到的MJPG帧需要在TCP线程中读,在Grab操作线程中写,这种被多个线程访问的资源需要加锁防止读写冲突,即资源被Grab操作写入时,需要上锁,不允许其它线程访问,操作完成时需要解锁,允许其它线程访问: - pthread_mutex_lock(&pmt);
/ a0 A" u( U3 ^' G2 b% _: c' I' r6 C" w6 A - pic_tmpbuffer = pic.tmpbuffer;. h8 \+ X$ n. I8 T1 k( x) \3 o
- pic.tmpbytesused = buff.bytesused;
; }/ A% {# Z6 ` - pic_tmpbytesused = pic.tmpbytesused;
) i8 K! V: \( L - pthread_cond_broadcast(&pct);
/ @6 L" c* ~, o6 I# J% b" S - pthread_mutex_unlock(&pmt);
复制代码线程互斥锁使用之前需要初始化: - pthread_mutex_t pmt;2 M& X) Y. l/ I- ^' n) J
- pthread_cond_t pct;, d4 w" `/ {; v
- int main(int argc, char* argv[])
, C1 C! f7 I/ i$ d+ d5 R - {; d+ C/ X- B, a! ~8 v
- ...
9 q) v$ s6 Z8 ^5 N2 `/ [7 ]1 Z - TCP_Server_Found(&socket_web_server , (char*)argv[2] , PORT_TCP);
. m' @9 _# i! X& c a0 _: N- v: g/ n - pthread_mutex_init(&pmt , NULL);( r* E" o1 n7 i. }3 c _
- pthread_create(&tid_tcp_web_recv , NULL , Thread_TCP_Web_Recv , NULL);
/ ~- V6 K. s+ A# H - pthread_create(&tid_tcp_web_send , NULL , Thread_TCP_Web_Send , NULL); ?! P$ b+ D! D5 }" Y* Q% o0 }
- ...8 h( `% Z. t9 o# _* r$ |8 q1 Q
- while(1)6 S: {) S% {5 I, v
- {
6 a! W4 ]: f7 \7 b8 _( G% F - V4l2_Grab_Mjpeg(false , MJPEG_FILE_NAME);$ X: _' H3 |% N5 i, Q& s, c2 |
- .../ {+ C9 ?( t! Q+ m X
- }- {: M. t6 ?5 m; q# B
- ...
, D# o. R0 m" m; S/ j - }
复制代码然后是发送的细节,发送图片文件之前,需要先发送HTTP标准头,这个相当于给发送图片或者其它类型的流数据铺路: - <p style="box-sizing: border-box; border: 0px; vertical-align: baseline; line-height: 26px;"><span style="text-indent: 32px;">
) Y$ h3 H7 u. S7 a& a3 |$ ` - </span></p><p style="box-sizing: border-box; border: 0px; vertical-align: baseline; line-height: 26px;"></p>
复制代码- #define STD_HEADER "Connection: close\r\n" \3 D, q$ ?( S0 ^5 ]
- "Server: MJPG-Streamer/0.2\r\n" \
4 U$ W# N# B( a7 u' h- M - "Cache-Control: no-store, no-cache, must-revalidate, pre-check=0, post-check=0, max-age=0\r\n" \9 ]7 H# g$ |: r& {% C8 M" j
- "Pragma: no-cache\r\n" \
4 `+ O1 G0 F) V/ I - "Expires: Mon, 3 Jan 2000 12:34:56 GMT\r\n"
) x$ T- N/ Q% M* A# y- [ - #define BOUNDARY "boundarydonotcross"/ @' {7 D3 _( H$ k. b
- printf("preparing header\n");8 H# F- D; G% B/ ~7 P2 b
- sprintf(buffer, "HTTP/1.0 200 OK\r\n" \
- T7 l, _$ `" K$ i+ x6 M - "Access-Control-Allow-Origin: *\r\n" \
; @6 |8 l) E3 | - STD_HEADER \& I- f$ v0 s6 I- K4 d( ?
- "Content-Type: multipart/x-mixed-replace;boundary=" BOUNDARY "\r\n" \/ z8 T1 Y+ ~3 |! w: B
- "\r\n" \
- V# _- G8 q; S7 z8 U - "--" BOUNDARY "\r\n");! n% I8 v! m4 K
- if(write(fd, buffer, strlen(buffer)) < 0)
. P9 g$ X6 g3 I2 U9 [) a. z - {0 O: L7 k; c' O8 g8 ]9 n0 g
- free(frame);
8 a2 w8 o# n8 v* {# } - return;. A5 J& L& ]) i5 h
- }
复制代码发送完HTTP标准头之后,就需要发送内容头(Content-Type),这处的Content-Type为image/jpeg,同样,HTTP标准协议里面image支持的类型远不止jpeg一种,发送完内容头之后就是正文和boundary结尾,这样帧完整的HTTP头发送到指定的TCP GET地址,就会在浏览器中显示刚刚发送的图片: - <pre class="prettyprint lang-cpp" style="box-sizing: border-box; font-family: Monaco, Menlo, Consolas, "Courier New", monospace; font-size: 16px; white-space: pre-wrap; line-height: 1.38462; color: rgb(51, 51, 51); word-break: break-all; background-color: rgb(245, 245, 245); border: 0px; border-radius: 4px; vertical-align: baseline;"> sprintf(buffer, "Content-Type: image/jpeg\r\n" \" I. s3 B, T3 U K. c
- "Content-Length: %d\r\n" \
% U7 J9 k3 D& n2 B - "X-Timestamp: %d.%06d\r\n" \" C( b3 P( X4 ^/ p9 {
- "\r\n", frame_size, (int)timestamp.tv_sec, (int)timestamp.tv_usec);0 o% }! V: c8 {- |6 a. a' b& d# ?
- printf("sending intemdiate header\n");
4 ]: x: E1 Q* `8 e. ~ - if(write(fd, buffer, strlen(buffer)) < 0)% X$ P) Q7 o, {$ x% W' B4 z1 ]5 G
- break;
' Q6 ]4 @# H, G& V" D0 o. k' f* u7 e - printf("sending frame\n");
6 f2 x& J0 i, }/ P o% H - if(write(fd, frame, frame_size) < 0)
1 L- T$ a( C3 b - break;
( |# `( n! _6 f# `: Z/ M - printf("sending boundary\n");
+ U) E" E0 l( z% `" ~" S8 ?2 ?. Z6 F - sprintf(buffer, "\r\n--" BOUNDARY "\r\n");1 Z! _7 |( s7 ]" x! T3 ~
- if(write(fd, buffer, strlen(buffer)) < 0)
: ]1 \$ B0 Y* E8 S* j4 d$ M2 f! v - break;</pre><p style="box-sizing: border-box; border: 0px; font-size: 16px; vertical-align: baseline; line-height: 26px; color: rgb(51, 51, 51); font-family: " helvetica="" neue",="" helvetica,="" tahoma,="" arial,="" "microsoft="" yahei",="" "hiragino="" sans="" gb",="" "wenquanyi="" micro="" hei",="" sans-serif;"=""></p>
复制代码另外需要说明的是,TCP服务器线程在发送MJPEG流的时候是死循环发送的,因此TCP客户端在发送完GET指令之后,就会收到TCP服务器循环发送的图像缓存,TCP客户端会陷入忙等待状态无法再对外发送任何GET或者POST指令,从客户端使用者角度来看的效果就是网页一直在等待。  - I5 K" z1 } j; M6 @) n
 " [% }9 L) z: d1 H& X- f6 C
二、UDP上位机UDP发送操作,同样需要先建立UDP Socket:
/ C9 n+ `' K# o- int UDP_Send_Found(socklen_t* socket_found , struct sockaddr_in *addr , char* ip , int port)) m/ ]3 Q0 T7 {/ G& O3 j
- {
2 g" p7 g! P. n - *socket_found = socket(AF_INET, SOCK_DGRAM, 0);
9 Q! K3 c: b1 B/ b9 |5 T - if(*socket_found == (~0))
+ ^+ d2 h" E3 K( W" @4 } - {
J* A. E! v0 t6 C - printf("Create udp send socket failed!\n");7 P0 J0 d m1 J# b
- return -1;
) l$ Y, V' {; w, r; q/ d - }+ E- U) o& r. U% v
- addr->sin_family = AF_INET;& I( r; f8 J0 X& e' [; h. @8 V
- addr->sin_addr.s_addr = inet_addr(ip);7 z- o7 Z3 A. m$ `
- addr->sin_port = htons(port);
+ Z U! B: P6 _2 i - memset(addr->sin_zero, 0, 8);% |6 R$ W6 U, I9 \ r: H2 Q
- return 0;. g6 g- Y+ t9 b
- }
复制代码
' h( I0 G7 J9 @' d4 z
4 Y7 A/ \- A- c2 Y8 k而UDP文件发送则要比HTTP发送简单得多,只需要将文件切片,每一片为固定长度的UDP帧长度,逐帧发送即可:
; n! D& E% I' F8 E; i, e' T$ t2 y; k
/ J* {2 C( m" E% c
! p3 o0 H8 M5 J6 i% x- while(fend > 0)7 o# f" A: S3 g8 J; i& ]! [: O
- {
0 h, b+ j6 P. `2 q1 z, R - memset(picture.data , 0 , sizeof(picture.data));# I2 g- [9 W% {$ ~
- fread(picture.data , UDP_FRAME_LEN , 1, fp);
9 }1 B! ?" C9 c - if(fend >= UDP_FRAME_LEN)
1 ?' ], M( e! }2 x9 W( f3 p - {
$ i% t( V. N5 i& ~ - picture.length = UDP_FRAME_LEN;
. V% r6 W4 d7 D - picture.fin = 0; f! v3 t' A) p% ~' F8 h2 t2 H- a" i
- }
1 J! d% w! p$ S' \# \+ R% a - else1 e% R1 l L) k3 G
- {
( J; R6 R2 F' S1 z - picture.length = fend;% i2 X v3 a! d( H5 S y" f
- picture.fin = 1;: I3 y! }) i$ P1 i: q
- }8 o8 B7 O* O" U8 ]' j7 u0 N' @
- //printf("sendbytes = %d \n",sendbytes);
9 e6 ]7 x) b& T* k8 Z+ f - sendbytes = sendto(socket_send, (char *)&picture, sizeof(struct Package), 0, (struct sockaddr*)&addr,addr_len);7 U7 V; o2 k; e# [
- if(sendbytes == -1) \: D5 f3 }$ h) m0 Q' r. K
- {4 C7 } V" p L1 F. H* e$ q) b- F
- printf("Send Picture Failed!d\n");
- E/ |% o* ^3 s - return -1;
, I4 c/ t5 @. X - }; {6 k4 U+ H) h* b- ^* Q0 z
- else
1 [" z# g" y/ V7 ~+ F - {& |5 t; c5 d: F# b
- fend -= UDP_FRAME_LEN;
8 ^+ ?1 I: e0 a# p6 D9 A! \: P/ G - }
' ]$ ]6 q: {- ~' ~1 S - }
复制代码 ; m/ c I' N, I/ e0 E4 L
. } N% _* ^# u( B
 6 a# O# o0 N( W Z% K" A
0 C! T6 j! ` G+ P: F6 FiMX8MPlus 核心板: https://www.forlinx.com/product/136.html |