本帖最后由 飞凌-marketing 于 2022-1-27 11:00 编辑
/ r2 g( }+ V% |' _6 {* `2 N
% E" F* A4 G* F! B6 S" `: @ 2 L5 ~6 p" d* L/ P5 |0 S5 X9 [
作者|donatello1996 来源 | 电子发烧友 题图|飞凌嵌入式 iMX8MPlus 核心板: https://www.forlinx.com/product/136.html6 Y8 |0 s4 w6 }7 C
- ^( {9 Z/ P% t. { _本文采用的硬件板卡为飞凌嵌入式OKMX8MP-C开发板,系统版本Linux5.4.70+Qt5.15.0,主要介绍基于HTTP网页服务器和UDP上位机的MJPG码流传输。 MJPG格式作为一种持续传输的视频码流,在远程监控领域中应用较广,而实现这种远程监控的第三方应用最常见的有两种:浏览器HTTP网页、UDP上位机。 ) j1 ~% s3 ~. p, ?) ]; K, C3 R
 ! b, k1 l/ A* d4 [- G, s
两者各有优势,对比鲜明,其中: 这两种应用各有优缺点,对于嵌入式开发者来说,两者都必须掌握。
. i p& w' C# W" |$ w0 M2 ~% O; W一、HTTP网页服务器9 t9 @: z3 T( H" I3 u' q
先说下HTTP网页服务器获取MJPG码流的代码,首先是OKMX8MP-C在开发板端建立TCP服务器: - int TCP_Server_Found(socklen_t* socket_found , char* ip , int port)! J; \# Y. e5 N
- {
! ^& I" z3 g/ E2 i7 ?9 K3 h - struct sockaddr_in servaddr;
9 c2 B( p: A/ D* F- B3 _ - socklen_t addrsize = sizeof(struct sockaddr);/ u Y+ f1 C: g% W- W
- bzero(&servaddr , sizeof(servaddr)); n( {9 g4 O; h* E( X, G
- servaddr.sin_family = AF_INET;: C. w4 k$ c0 T3 C9 n
- servaddr.sin_addr.s_addr = inet_addr(ip);5 p4 g9 t+ a' `7 o# b: Z* e
- servaddr.sin_port = htons(port);* G/ \% N5 g; G0 m- z
- int ret;, m( _, x9 R+ L1 t, L2 U* z
- IF( (*socket_found = socket(AF_INET , SOCK_STREAM , 0)) == -1)4 m% `( _" V; q# R' ^/ G
- {! b. |* g5 e% p9 U7 p3 x
- printf("Create socket error: %s (errno :%d)\n",strerror(errno),errno);7 y/ J1 R1 r$ h
- return -1;
" \1 U, w, D. f' c$ J1 X - }5 x. z) J' t1 M) f
- int on = 1;; {6 O; V; d8 ~
- if(setsockopt(*socket_found , SOL_SOCKET, SO_REUSEADDR, &on, sizeof(on)) < 0)
, ~& q+ @, N# {0 n6 l - {
! x! f* G& o' ^$ d) E( f - printf("setsockopt error\n");. r4 O$ e. G- ^- x% S
- }/ K) N- z, W1 C2 B
- ret = bind(*socket_found , (struct sockaddr *)&servaddr , addrsize);
4 |* K G# C1 N! m, K8 z% K- u - if(ret == -1)) Y6 b" x1 h* w
- {6 K2 g( V7 }2 y
- printf("Tcp bind faiLED!\n");
. o- h0 b7 d7 V5 @6 I! C - return -1;6 y' u2 Z n' J+ O( P; G
- }, d5 ~" Q8 f$ r
- if(listen(*socket_found , 5) == -1)+ J R, G3 B& }! O6 n) L6 B
- {
! H' |* g! n7 |. i8 q - printf("Listen failed!\n");7 y5 G( y3 g$ c7 i) Y9 }3 `0 k
- return -1;
! X0 V8 Z" F; F* O - }- B8 D- u# T) a5 H0 r
- return 0;
3 M% {7 x. d1 S7 o$ D - }
复制代码其中setsockopt()函数是可选的,一般只用于规避socket()函数的建立错误。 建立了TCP服务器后,返回的socklen_t型实参在后面的HTTP网页服务器中需要用到。 HTTP网页服务器所属的TCP操作是需要另起轮询线程来让客户端进行accept()握手操作的,accept()之前的listen()倒是只需要执行一次即可,accept()握手操作和recv()接收操作需要创建一个死循环线程: - pthread_create(&tid_tcp_web_recv , NULL , Thread_TCP_Web_Recv , NULL);
9 L3 H2 j- u+ ?" s# E* Z - void * Thread_TCP_Web_Recv(void *arg)
1 ^2 d: U8 M; s0 k8 H( f% ] - {. `/ k: h6 F1 ~# u" u# I
- 。。。! |3 p$ w7 o; q7 R3 b
- while(1)9 t. g9 Z, V$ L9 m# b4 _
- {. L, f) a: ~! [! Q: H6 g
- fd_socket_conn = accept(socket_web_server , (struct sockaddr *)&sockaddr_in_conn , &addrsize);
+ U# G0 R8 X! X, r. }% E - printf("fd_socket_conn = accept()\n");
0 S4 q5 j0 u4 {# J* [& @" s* c# X* z - 。。。
) B4 K& L e2 o) e) I8 j4 F - recv(fd_socket_conn , recvbuf , 1000 , 0);4 Q6 c6 u. `$ o" i
- }& C! E; y$ _" M* r$ @
- 。。。- T3 z, P- O/ Y2 _
- }
复制代码MJPG帧可以使用Grab操作获取,获取到的MJPG帧需要在TCP线程中读,在Grab操作线程中写,这种被多个线程访问的资源需要加锁防止读写冲突,即资源被Grab操作写入时,需要上锁,不允许其它线程访问,操作完成时需要解锁,允许其它线程访问: - pthread_mutex_lock(&pmt);
. B" y/ j o$ r1 a, G - pic_tmpbuffer = pic.tmpbuffer;
1 q( q& s6 Z: q: x5 b% y t - pic.tmpbytesused = buff.bytesused;' j- V2 e4 t7 k
- pic_tmpbytesused = pic.tmpbytesused;
1 T" O/ G4 X+ s( ~/ g5 U - pthread_cond_broadcast(&pct);! d+ k! G/ I# x3 q, m5 A0 I
- pthread_mutex_unlock(&pmt);
复制代码线程互斥锁使用之前需要初始化: - pthread_mutex_t pmt;1 F. g4 M2 t. n8 V- x
- pthread_cond_t pct;
" K; _' K$ U O' E3 H/ n7 s - int main(int argc, char* argv[])
" W: ]% K( I( O' ? R2 q - {
- |9 R8 [+ b5 z+ H9 w n: ~3 J P( }& L9 B - ...
* i3 ~* z' [9 M8 t- Z - TCP_Server_Found(&socket_web_server , (char*)argv[2] , PORT_TCP);
9 h" g1 u/ ~* ]4 i+ z - pthread_mutex_init(&pmt , NULL);. C( Y- n! X4 A, c: M
- pthread_create(&tid_tcp_web_recv , NULL , Thread_TCP_Web_Recv , NULL);
2 I, B6 J" u7 p+ O2 {5 C) C; u) P { - pthread_create(&tid_tcp_web_send , NULL , Thread_TCP_Web_Send , NULL);: d0 |5 R+ p- T$ c" \
- ...- \( e6 I$ l9 W5 ]1 u
- while(1)/ Y* C* e/ w/ h( i
- {( A% Q( s( \: B% g
- V4l2_Grab_Mjpeg(false , MJPEG_FILE_NAME);3 W! e* a7 z# \6 R2 b* i
- ...( o8 z+ Q$ p& e5 [, ^0 l3 X1 `
- }
4 h& T9 {9 S% L - ...( P- M' U; Z/ F' J2 i1 V( E+ h
- }
复制代码然后是发送的细节,发送图片文件之前,需要先发送HTTP标准头,这个相当于给发送图片或者其它类型的流数据铺路: - <p style="box-sizing: border-box; border: 0px; vertical-align: baseline; line-height: 26px;"><span style="text-indent: 32px;">
; S$ v5 Z8 k( _8 U+ L# O - </span></p><p style="box-sizing: border-box; border: 0px; vertical-align: baseline; line-height: 26px;"></p>
复制代码- #define STD_HEADER "Connection: close\r\n" \6 g, M7 r& w1 z+ Y- s+ n2 G
- "Server: MJPG-Streamer/0.2\r\n" \
, }0 b" I, S0 |9 L) N* ] - "Cache-Control: no-store, no-cache, must-revalidate, pre-check=0, post-check=0, max-age=0\r\n" \
0 t7 Y# H9 n# d0 e" O7 V - "Pragma: no-cache\r\n" \0 M t+ F, w& f5 y
- "Expires: Mon, 3 Jan 2000 12:34:56 GMT\r\n"! Y( a7 O1 u, z8 X, ^- z+ }: C
- #define BOUNDARY "boundarydonotcross", M$ a; }* b* U! }1 o
- printf("preparing header\n");
& y, z; C# P( D+ u; x - sprintf(buffer, "HTTP/1.0 200 OK\r\n" \
8 g0 o0 w+ C* Z% h - "Access-Control-Allow-Origin: *\r\n" \4 g) w( N" }) k: V" E8 K
- STD_HEADER \
4 D% T, h" P$ F - "Content-Type: multipart/x-mixed-replace;boundary=" BOUNDARY "\r\n" \
* ], Y# S! q9 E - "\r\n" \' Q& E$ V2 o" i
- "--" BOUNDARY "\r\n");
9 r" |9 f9 f6 c - if(write(fd, buffer, strlen(buffer)) < 0)! q/ [, [% T; I( o2 z0 N0 v
- {
s# r/ u" ?% Z0 b, @/ I! _- D$ Y - free(frame);2 B& G- w- [- W. r7 k# X
- return;
; z$ b( u- `1 S# F8 [6 o% n& N" E P - }
复制代码发送完HTTP标准头之后,就需要发送内容头(Content-Type),这处的Content-Type为image/jpeg,同样,HTTP标准协议里面image支持的类型远不止jpeg一种,发送完内容头之后就是正文和boundary结尾,这样帧完整的HTTP头发送到指定的TCP GET地址,就会在浏览器中显示刚刚发送的图片: - <pre class="prettyprint lang-cpp" style="box-sizing: border-box; font-family: Monaco, Menlo, Consolas, "Courier New", monospace; font-size: 16px; white-space: pre-wrap; line-height: 1.38462; color: rgb(51, 51, 51); word-break: break-all; background-color: rgb(245, 245, 245); border: 0px; border-radius: 4px; vertical-align: baseline;"> sprintf(buffer, "Content-Type: image/jpeg\r\n" \4 v& a% f% U- F
- "Content-Length: %d\r\n" \2 R# A. ^. J7 T: X3 ~0 {
- "X-Timestamp: %d.%06d\r\n" \
# c0 I+ o d( ~* ? - "\r\n", frame_size, (int)timestamp.tv_sec, (int)timestamp.tv_usec);* `4 @* h0 K9 g4 X6 c9 D; }$ [
- printf("sending intemdiate header\n");
0 x7 t* O) h: d9 h2 a* t# m - if(write(fd, buffer, strlen(buffer)) < 0)9 l& W! [- ~1 X" q0 X7 ]) b
- break;4 @: L4 ]5 q. J0 ?) j+ B3 S
- printf("sending frame\n");( [1 X8 C% V% y5 |: R Y A
- if(write(fd, frame, frame_size) < 0) L/ V- |6 J! X" ?' r p0 R, T
- break;6 X% @- ]8 k' v: n( X U
- printf("sending boundary\n");4 L1 I! g) V3 O* }8 ^% e
- sprintf(buffer, "\r\n--" BOUNDARY "\r\n"); ^2 I% o2 H6 G7 @. a6 y: D z; y) P
- if(write(fd, buffer, strlen(buffer)) < 0)
$ S8 v4 X. `9 X& N* @- o6 W - break;</pre><p style="box-sizing: border-box; border: 0px; font-size: 16px; vertical-align: baseline; line-height: 26px; color: rgb(51, 51, 51); font-family: " helvetica="" neue",="" helvetica,="" tahoma,="" arial,="" "microsoft="" yahei",="" "hiragino="" sans="" gb",="" "wenquanyi="" micro="" hei",="" sans-serif;"=""></p>
复制代码另外需要说明的是,TCP服务器线程在发送MJPEG流的时候是死循环发送的,因此TCP客户端在发送完GET指令之后,就会收到TCP服务器循环发送的图像缓存,TCP客户端会陷入忙等待状态无法再对外发送任何GET或者POST指令,从客户端使用者角度来看的效果就是网页一直在等待。 
; n9 M: c9 f+ @) ?( t' M$ { 3 u) J. @" F1 W8 S* I
二、UDP上位机UDP发送操作,同样需要先建立UDP Socket:
Y7 X2 ?+ x/ B' S6 I- int UDP_Send_Found(socklen_t* socket_found , struct sockaddr_in *addr , char* ip , int port)
; O3 u' {& Y( z r. g x; G - {
. k, \( U$ y6 U - *socket_found = socket(AF_INET, SOCK_DGRAM, 0);
3 c5 }& X9 h1 T9 y4 u8 b - if(*socket_found == (~0))+ t6 r$ Y7 u7 F4 ^! P0 Q8 N
- {8 r$ g( M! t" m* i3 O! ~$ {+ N( X0 F4 u
- printf("Create udp send socket failed!\n");
+ w4 d- G* n0 C, {- v6 W$ j3 W* q - return -1;* q- c5 W8 j1 F7 e0 o2 {0 M
- }
& P. ?& x$ Y; q7 a# C- \ - addr->sin_family = AF_INET;
( U1 g! E; V" e' X - addr->sin_addr.s_addr = inet_addr(ip);8 q7 T3 Z, R6 b; R O' u
- addr->sin_port = htons(port);1 I( V+ U H6 A L( g# L
- memset(addr->sin_zero, 0, 8);
4 H, ^! ~2 ?; e - return 0;1 n Q: g, H, F6 ]8 O% i
- }
复制代码 6 |8 D0 {9 r3 r: {8 z; t7 u
7 L4 b: X( D" v1 m% j5 V
而UDP文件发送则要比HTTP发送简单得多,只需要将文件切片,每一片为固定长度的UDP帧长度,逐帧发送即可:+ F- t( H( I9 f) r; ]) k$ W
* b; j, B, N' A8 d
7 |. W& c7 N) d' |7 R& p. C8 T( I- while(fend > 0)' K8 G: f i" b6 E+ L! A' u
- {
9 N& t) z) e3 E$ w( f8 F. l1 [, m& t - memset(picture.data , 0 , sizeof(picture.data));
7 Z7 H/ {0 `" b* h' M& a$ B - fread(picture.data , UDP_FRAME_LEN , 1, fp);% g7 P. M$ Z; X. ]' r/ f" ~
- if(fend >= UDP_FRAME_LEN)% I+ p1 R/ g: n6 h
- {
8 m% w3 r0 F" \1 w* u - picture.length = UDP_FRAME_LEN;, t; u0 e* o8 a5 a# Q2 Z
- picture.fin = 0;
4 I3 x4 T& l$ | L6 g- b - }
6 l/ w5 ~+ |9 k7 p( @7 N- j - else; E; _2 Q1 u. o9 w: a
- {
' c* z- n9 w/ m7 j! O - picture.length = fend;: f) w% U/ o' H* p
- picture.fin = 1;
, r! v" r2 ]) p2 l6 y - }
! e( D2 e# \- K* k7 F- q - //printf("sendbytes = %d \n",sendbytes);
. a# z7 T6 m3 e( y7 C4 ] - sendbytes = sendto(socket_send, (char *)&picture, sizeof(struct Package), 0, (struct sockaddr*)&addr,addr_len);$ H/ ?" q) j& }& a" ?4 z) ^) @" Q
- if(sendbytes == -1)9 m1 p8 U6 c' v i: l
- {: M9 T q# \2 u0 _( G
- printf("Send Picture Failed!d\n");$ t* t( c) @, a
- return -1;7 r4 v1 w: O$ h; ~- Z7 G4 ]
- }7 k4 n3 f j3 _, Y7 o
- else. u. [/ e5 T) {! b
- {
- [6 E* j9 O/ U7 M! Q9 h - fend -= UDP_FRAME_LEN;
, e! j* I/ W9 z7 N3 o3 P3 d - }* I3 |% u6 R, e0 \
- }
复制代码
* [& `* Q. Q4 K: X7 j( y( J. y2 w2 f5 e5 K: b

+ X } f) k1 ?8 V
$ L* j) b: `' {( ?iMX8MPlus 核心板: https://www.forlinx.com/product/136.html |