本帖最后由 飞凌-marketing 于 2022-1-27 11:00 编辑
6 s; R( w2 s; O6 f! h
8 Y- u" E" T0 o( B. M5 d* J
8 M9 F; \* ?1 C3 x0 `作者|donatello1996 来源 | 电子发烧友 题图|飞凌嵌入式 iMX8MPlus 核心板: https://www.forlinx.com/product/136.html' d; e! e6 k$ i$ h% V+ d
% _9 C u' g& ^4 q本文采用的硬件板卡为飞凌嵌入式OKMX8MP-C开发板,系统版本Linux5.4.70+Qt5.15.0,主要介绍基于HTTP网页服务器和UDP上位机的MJPG码流传输。 MJPG格式作为一种持续传输的视频码流,在远程监控领域中应用较广,而实现这种远程监控的第三方应用最常见的有两种:浏览器HTTP网页、UDP上位机。 ' D0 E n- w* ]3 x

2 q) d/ H8 S+ ` q两者各有优势,对比鲜明,其中: 这两种应用各有优缺点,对于嵌入式开发者来说,两者都必须掌握。 8 E6 d5 m. T* b+ J
一、HTTP网页服务器
0 A4 J9 {# [ T( {- z先说下HTTP网页服务器获取MJPG码流的代码,首先是OKMX8MP-C在开发板端建立TCP服务器: - int TCP_Server_Found(socklen_t* socket_found , char* ip , int port)
% r0 P4 Z" X. I. ~# {9 W$ ` - {
% x) I9 y4 h9 I. L" q3 f& u2 r - struct sockaddr_in servaddr; \+ Y& L1 }. c3 U t
- socklen_t addrsize = sizeof(struct sockaddr);/ B$ ?8 y1 [1 j1 H3 H5 E
- bzero(&servaddr , sizeof(servaddr));5 O& a8 V! |* |# b. E) f
- servaddr.sin_family = AF_INET;
! n9 E3 {) ?' L7 ~, b - servaddr.sin_addr.s_addr = inet_addr(ip);
. N H) J' b. N6 P9 n2 C! X- ]2 Y7 ?1 l - servaddr.sin_port = htons(port);
6 l* O* M4 k- o$ i) S - int ret;
) ~$ l% P3 c8 c9 H* }: {7 {/ W - IF( (*socket_found = socket(AF_INET , SOCK_STREAM , 0)) == -1)
' b4 o9 D. A8 F: H% k - {- Y8 g( g9 Y8 ]* k& |' Y9 i* K; O
- printf("Create socket error: %s (errno :%d)\n",strerror(errno),errno);
2 T; r% |2 f2 V6 T0 A! g/ ^ - return -1;
* ~3 F. m- P. _) F( d - }
2 m7 Z. B4 ~* D! X1 a/ N7 b - int on = 1;* e' t6 M% D L( R* x
- if(setsockopt(*socket_found , SOL_SOCKET, SO_REUSEADDR, &on, sizeof(on)) < 0)
0 k; X$ ?& _! ^( M" w* s - {# |! U; J; o3 {" l8 T
- printf("setsockopt error\n");3 W. C* p* C/ E4 e
- }
/ s! n3 G. g8 c9 c - ret = bind(*socket_found , (struct sockaddr *)&servaddr , addrsize);
; e" P+ a5 j& G& t5 Y5 L; @ - if(ret == -1)1 d& S" D* Y0 d: o6 `. z* h8 H
- {* B4 V o$ _! e2 U
- printf("Tcp bind faiLED!\n");; p4 \6 Y- ]* } {5 C4 y! n
- return -1;
4 S2 D: q4 ?4 \1 z5 O - }/ {! @; d$ I8 T7 _4 P2 z
- if(listen(*socket_found , 5) == -1)
$ B- F m: V, A" Z A8 j- L$ z - {8 m* ~* k% x' R4 J6 Y
- printf("Listen failed!\n");
. V' o F8 f, M$ x! l/ w+ } - return -1;
& u0 q) z9 ]0 B) @, F5 Q - }) Y& H" V, E2 r1 |1 g4 @1 {
- return 0;/ F2 q; J' d2 C: ~: v
- }
复制代码其中setsockopt()函数是可选的,一般只用于规避socket()函数的建立错误。 建立了TCP服务器后,返回的socklen_t型实参在后面的HTTP网页服务器中需要用到。 HTTP网页服务器所属的TCP操作是需要另起轮询线程来让客户端进行accept()握手操作的,accept()之前的listen()倒是只需要执行一次即可,accept()握手操作和recv()接收操作需要创建一个死循环线程: - pthread_create(&tid_tcp_web_recv , NULL , Thread_TCP_Web_Recv , NULL);+ ?5 l. x* G9 f/ `0 X, ?0 `6 Q
- void * Thread_TCP_Web_Recv(void *arg)& |6 \9 M9 s8 }7 d3 e W1 a/ h
- {
* J9 t$ ?; H" `/ l - 。。。: @# [' H" X) }4 N M
- while(1)
. c; M0 w2 X. u+ H" v8 I! o8 I: \4 I - {
1 _, a: u5 g/ | - fd_socket_conn = accept(socket_web_server , (struct sockaddr *)&sockaddr_in_conn , &addrsize);
. Q1 X6 t3 ^4 C6 ]( K - printf("fd_socket_conn = accept()\n");8 ?, Y: }# v$ q" } Z
- 。。。
( T: O0 z6 M6 p+ `4 y" x - recv(fd_socket_conn , recvbuf , 1000 , 0);; k! y6 {5 I3 l- ~& D. N6 W! H. `0 y
- }
- t9 X. m2 p! q0 h- w) v% e - 。。。0 d) ^0 i$ ?3 o3 u0 U$ e3 l a
- }
复制代码MJPG帧可以使用Grab操作获取,获取到的MJPG帧需要在TCP线程中读,在Grab操作线程中写,这种被多个线程访问的资源需要加锁防止读写冲突,即资源被Grab操作写入时,需要上锁,不允许其它线程访问,操作完成时需要解锁,允许其它线程访问: - pthread_mutex_lock(&pmt);
5 [1 [8 W; b4 b5 p2 Z' _ - pic_tmpbuffer = pic.tmpbuffer;4 N7 N% B; \7 Z7 o, b# Y) Y& D6 \
- pic.tmpbytesused = buff.bytesused;
7 Y- L2 k) j- L& u - pic_tmpbytesused = pic.tmpbytesused;
/ n+ i3 z% D- I; E - pthread_cond_broadcast(&pct);
9 @1 \7 A! p' o# N - pthread_mutex_unlock(&pmt);
复制代码线程互斥锁使用之前需要初始化: - pthread_mutex_t pmt;0 ^; Y5 y5 }4 Z, F9 C5 q
- pthread_cond_t pct;0 R0 l; a6 X: H( |4 f
- int main(int argc, char* argv[])
# c$ [( A0 |; F2 I7 a - {* ]# b! f. Q' M
- ...5 c3 w' X! j) v( S
- TCP_Server_Found(&socket_web_server , (char*)argv[2] , PORT_TCP);% w, s4 v" k0 |" Q* R
- pthread_mutex_init(&pmt , NULL);7 E# W. ^4 S' @6 y" A0 f6 M5 U& x
- pthread_create(&tid_tcp_web_recv , NULL , Thread_TCP_Web_Recv , NULL);+ z, ~1 v- L w$ H. A( w
- pthread_create(&tid_tcp_web_send , NULL , Thread_TCP_Web_Send , NULL);4 D* ~) U' W9 N- _/ P/ n
- ...# t ?5 E d9 Y9 f
- while(1)
- {- E" P: E3 o/ E# n - {3 m: D8 @3 b A% j& S. E. V; ~
- V4l2_Grab_Mjpeg(false , MJPEG_FILE_NAME);" f& i. i; q* r3 ]" q- d4 A: I0 a
- ...
5 D+ I1 G9 v+ a) n- ` - }
3 m: q) \# ?. }2 ? - ...
- w, u( e8 H3 O4 W5 [2 [ - }
复制代码然后是发送的细节,发送图片文件之前,需要先发送HTTP标准头,这个相当于给发送图片或者其它类型的流数据铺路: - <p style="box-sizing: border-box; border: 0px; vertical-align: baseline; line-height: 26px;"><span style="text-indent: 32px;">
3 e* u! M: @3 \# q - </span></p><p style="box-sizing: border-box; border: 0px; vertical-align: baseline; line-height: 26px;"></p>
复制代码- #define STD_HEADER "Connection: close\r\n" \/ y" Z/ W) w+ |- Q
- "Server: MJPG-Streamer/0.2\r\n" \
/ x5 u3 `6 N* P: { - "Cache-Control: no-store, no-cache, must-revalidate, pre-check=0, post-check=0, max-age=0\r\n" \
% R: A5 L% [6 B - "Pragma: no-cache\r\n" \
, ^! J$ J5 h9 y1 h - "Expires: Mon, 3 Jan 2000 12:34:56 GMT\r\n"
- |! s6 l8 o) J" I, X! v - #define BOUNDARY "boundarydonotcross"1 R3 n, s; C# ]6 k) T+ p7 P
- printf("preparing header\n");
3 o. U! @6 f6 D8 c# W - sprintf(buffer, "HTTP/1.0 200 OK\r\n" \7 S9 F9 |! r- u0 |9 ~: `
- "Access-Control-Allow-Origin: *\r\n" \3 `8 \" o; R8 W4 U) {/ x$ M! l
- STD_HEADER \
9 Z9 M. |' T; h - "Content-Type: multipart/x-mixed-replace;boundary=" BOUNDARY "\r\n" \7 M6 p; v* x4 u" [" d
- "\r\n" \4 i* l' `9 v9 j3 K& W
- "--" BOUNDARY "\r\n");
" U- i) ^: |+ Y1 o$ y - if(write(fd, buffer, strlen(buffer)) < 0)8 ^* w$ C3 t' r& w Q( e8 ~
- {
" R5 I n1 m6 Q% i - free(frame);; u3 P4 [! G! y2 q+ ?8 c
- return;, j* s# M, z# B! [7 \
- }
复制代码发送完HTTP标准头之后,就需要发送内容头(Content-Type),这处的Content-Type为image/jpeg,同样,HTTP标准协议里面image支持的类型远不止jpeg一种,发送完内容头之后就是正文和boundary结尾,这样帧完整的HTTP头发送到指定的TCP GET地址,就会在浏览器中显示刚刚发送的图片: - <pre class="prettyprint lang-cpp" style="box-sizing: border-box; font-family: Monaco, Menlo, Consolas, "Courier New", monospace; font-size: 16px; white-space: pre-wrap; line-height: 1.38462; color: rgb(51, 51, 51); word-break: break-all; background-color: rgb(245, 245, 245); border: 0px; border-radius: 4px; vertical-align: baseline;"> sprintf(buffer, "Content-Type: image/jpeg\r\n" \# v" ?# X8 y: l+ p
- "Content-Length: %d\r\n" \0 S( d; U( a- M* X0 o. \" a2 s
- "X-Timestamp: %d.%06d\r\n" \
3 t$ B( Y( H8 k7 x1 `& t8 Q. U, o - "\r\n", frame_size, (int)timestamp.tv_sec, (int)timestamp.tv_usec); h6 E( L$ g3 w" X
- printf("sending intemdiate header\n");6 ^ f+ f; a; E$ p5 V$ V( s, t8 \
- if(write(fd, buffer, strlen(buffer)) < 0)( W3 q* p' `" W
- break;9 d! c) p6 Z Y
- printf("sending frame\n"); n& q+ G5 u2 v& r2 p% k
- if(write(fd, frame, frame_size) < 0)
" b* S+ O r, x: ^6 n2 \* R - break;
" {, L# K2 w+ s - printf("sending boundary\n");
9 q5 }- a" A/ d6 b: Z+ f, z! Q" Z - sprintf(buffer, "\r\n--" BOUNDARY "\r\n");2 u* G: w* W, [7 \' h, p
- if(write(fd, buffer, strlen(buffer)) < 0)' I4 B' I3 @0 ]
- break;</pre><p style="box-sizing: border-box; border: 0px; font-size: 16px; vertical-align: baseline; line-height: 26px; color: rgb(51, 51, 51); font-family: " helvetica="" neue",="" helvetica,="" tahoma,="" arial,="" "microsoft="" yahei",="" "hiragino="" sans="" gb",="" "wenquanyi="" micro="" hei",="" sans-serif;"=""></p>
复制代码另外需要说明的是,TCP服务器线程在发送MJPEG流的时候是死循环发送的,因此TCP客户端在发送完GET指令之后,就会收到TCP服务器循环发送的图像缓存,TCP客户端会陷入忙等待状态无法再对外发送任何GET或者POST指令,从客户端使用者角度来看的效果就是网页一直在等待。  . ?& X- B: I6 B3 [( e, R
 - Y3 Q- \. K/ d6 k5 H. b
二、UDP上位机UDP发送操作,同样需要先建立UDP Socket:" N4 [# Z5 s0 |2 p Q
- int UDP_Send_Found(socklen_t* socket_found , struct sockaddr_in *addr , char* ip , int port)
. R; O& n$ J6 w3 ^' k% N - {# `5 i+ Q q3 d' ]1 \2 e* }
- *socket_found = socket(AF_INET, SOCK_DGRAM, 0);1 A4 X" t$ _# i, F
- if(*socket_found == (~0))" p% P0 Y" J0 @+ L# J1 g: i
- {
4 u8 b. ^9 q* p, T, u9 B2 |2 f; c - printf("Create udp send socket failed!\n");
# j2 c' ^% s8 m- j- G+ H( A - return -1;
% H9 V; t% H3 b - }5 D- Q+ ?; K+ j+ @/ m# @' F( d
- addr->sin_family = AF_INET;/ s9 V: i5 \- n. R0 B
- addr->sin_addr.s_addr = inet_addr(ip);
& w6 V; l2 D! S' [ - addr->sin_port = htons(port);
7 J5 h3 V! N' P ?! s, K6 a - memset(addr->sin_zero, 0, 8);
" O% Z9 s0 t2 e% G - return 0;
9 P3 n( \, a& d* } - }
复制代码
) J2 Z+ u4 I a' I. U) u6 {4 U2 S/ y# }3 r4 A' Z
而UDP文件发送则要比HTTP发送简单得多,只需要将文件切片,每一片为固定长度的UDP帧长度,逐帧发送即可:
! r0 f% @: @ X' L; o
( g6 K& R. V! C8 O4 P- {4 u6 c8 B
9 n. Z8 u r7 g5 \+ [+ P ]- while(fend > 0)
& ]9 G: }; c, x- k! @ - {
. _3 l# `7 i5 W: ?8 |" ]& c - memset(picture.data , 0 , sizeof(picture.data));/ I1 R% w8 V1 x7 N; M5 ?
- fread(picture.data , UDP_FRAME_LEN , 1, fp);
6 W/ K; f5 H. a8 I - if(fend >= UDP_FRAME_LEN)
5 u! \1 P) J' Y* C3 V& F) v - {
3 i3 t0 n. }! @$ c7 N3 d! q - picture.length = UDP_FRAME_LEN;
" W' r# P9 H" B) i% \7 j8 A3 D - picture.fin = 0;
8 r; ^/ c" g% R - }
Z6 L1 f. N! I% g9 ` - else
0 h7 h& c W$ s: C! L1 d, _ - {
0 ~$ S5 Y, r, x5 i, B6 I/ e - picture.length = fend;
, _2 `+ ^' x0 o - picture.fin = 1;
4 b' M4 l& W' }3 { - }& B, m7 n- R- D
- //printf("sendbytes = %d \n",sendbytes);
: a, e' u8 }) k5 Z$ A4 L r - sendbytes = sendto(socket_send, (char *)&picture, sizeof(struct Package), 0, (struct sockaddr*)&addr,addr_len);% |, k# Q, v3 u$ J* Z- u
- if(sendbytes == -1)9 r. q" k, ^- h
- {
9 [4 O } Y: W& M - printf("Send Picture Failed!d\n");
; _% ~- ~5 [; s - return -1;8 @ b; d' Z" ^0 `5 ^# d5 g9 A& c
- }, g' [! Q% Z1 l G
- else, v' y4 G1 g8 c2 r+ `
- {. B L/ L! p! }' ^9 v
- fend -= UDP_FRAME_LEN;
/ B, ~! I7 g) d - }- }% n7 Y$ Z/ x9 [
- }
复制代码 " G3 J9 U$ k% D% D1 O
# |# x- a4 [( m8 z6 u7 `* F , A' e; d$ e$ S) O& k' j
$ i% \- m9 J# p. L% N6 C! x: X) T
iMX8MPlus 核心板: https://www.forlinx.com/product/136.html |