PNG File Format and Opportunities of Parallel Programming
Ruben Safir MS Comp Sci
Fall Semester 2017
Parallel Programming CS : Dr Samir
Resources:
|
Length |
4 Bytes in big endian format |
|
Type |
4 bytes |
|
Data |
0 - 2,147,483,647 bytes as defined in the Length field |
|
CRC |
4 byte |
Interpreting the chunks makes for the majority of the workload for programs to read and right. The second field is the TYPE field. The PNG specification has a long list of publicly defined TYPES although private data types as allowed for application developers.
TYPE Field:
The type field has 4 bytes which are associated with ASCII representations and are case sensitive. In general this is not a string, although a code can use this data as strings to make comparisons if they chose. Note, however, that there is no ending NULL and the data is Big Endian.
The Byte array distinguishes interpretation rules based on the capitalization of the individual bytes of the array (which can be quickly checked with a bit check of the 5th bit) The rules are detailed in the following chart
5th Bit (Capitalization) Interpretation of Chunk Types
|
Byte 0 |
Critical |
Is the byte critical for PNG interpretation |
|---|---|---|
|
Byte 1 |
Public |
Publicly Defined in the PNG Specification |
|
Byte 2 |
Reserved |
Must be always capitalized |
|
Byte 3 |
Safe to Copy |
Copy uninterpreted data if capitalized |
PNG Files has a minimum scaffolding that is necessary for a valid file.
PNG Header
IHDR Chunk – Header Information embedded in data field
IDAT Chunk – Actual Data that has to be decompressed and decoded
IEND – The End – you can check for this in order to stop
A common layout of the file might be like this:
PNG Signature
IHDR Chunk
PLTE Chuck <== Must come prior to the first IDAT header and there can only be one
IDAT … and repeated in serial to include all the image raw data
IEND
CRC is a check some which can be looked up in independent sources. Data is compressed with the zlib compression. It was developed to be a patent free, free software implementation for compression after several lawsuits over compression algorithms shocked the software word in the mid-1990’s. It is maintained by a cadres of free software developers for nearly 20 years and while there are opportunities for threading and parallel coding within the compression algorithm, its a huge detour from the core of this project and it is advised to use the standard zlib library deflate() for such decompression. I did want to say a word about its design. It works by sampling a window of a stream of data, and using that window for replacement words on the data incoming to the stream.
If you have a stream of bytes such as this with a window of 16 bytes
a a b b c c d d e e a a c d h h I I j j
the positions 11 and 12 can be replaced with data from 1-2 and position 13 and 14 (cd) can be replaced 7,8. The window then shifts with the stream.
a a b b c c d d e e % & h h I I j j
This gives flexibility in data streaming within the compression and allows for some threading since we can pick up the compression in the middle of the file.
A better location for parallel coding is within the IDAT headers themselves, since they are already divided into chunks of length as defined in the first 4 bytes of a chunk. In order to do that, we need to first examine and dissect a PNG file. I wrote code to do just this. It includes a .h header file, a .cpp library, a main function file, and a make file. The main file look as follows:
1 /*
2 * =====================================================================================
3 *
4 * Filename: main_png.cpp
5 *
6 * Description: PNG Project - main
7 *
8 * Version: 1.0
9 * Created: 11/30/2016 02:57:46 PM
10 * Revision: 1.0
11 * Compiler: gcc
12 *
13 * Author: Ruben Safir (mn), ruben@mrbrklyn.com
14 * Company: NYLXS Inc
15 *
16 * =====================================================================================
17 */
18 #include <iostream>
19 #include "png_proj.h"
20
21 int main(int argc, char **argv)
22 {
23 // const char * image_file_path = nullptr;
24 const char * image_file_path = "/home/ruben/images/tzfat/tzfat_dawn_blended_fused.png";
25 //const char * image_file_path = "/home/ruben/photo_album/images/135-red_sea_1.png";
26 png_proj::Image pngtestfile{image_file_path};
27 // pngtestfile.read_png();
28 std::cout << "MAIN: line:" << __LINE__ << " char * image size==>" << sizeof(image_file_path) << std::endl;
29 pngtestfile.read_header();
30 pngtestfile.read_chunk();
31 while(pngtestfile.getNext() < pngtestfile.get_end() ){
32 std::cout << "Hey I am not Kosher Bird, yoy can't eat me!!" << std::endl;
33 }
34
35 return EXIT_SUCCESS;
36 }The image is picked up and absorbed in RAM in whole
The header file, png_proj.h is as follows
1 /*
2 * =====================================================================================
3 *
4 * Filename: png_proj.h
5 *
6 * Description: png_proj.h
7 *
8 * Version: 1.0
9 * Created: 11/30/2016 03:50:13 PM
10 * Revision: none
11 * Compiler: gcc
12 *
13 * Author: Ruben Safir (mn), ruben@mrbrklyn.com
14 * Company: NYLXS Inc
15 *
16 * =====================================================================================
17 */
18
19 #ifndef PNGPRJ
20 #define PNGPRJ
21 #include <inttypes.h>
22 #include <iostream>
23
24 namespace png_proj{
25 typedef uint32_t CHUNK;
26 struct IHDR {
27 int32_t width;
28 int32_t height;
29 int8_t depth;
30 int8_t color_type; //0 greyscale, 2 RGB, 3 Palette, 4 greyscale with alpha, 6 RGB with alpha
31 int8_t compress; //must be 0
32 int8_t filter; // must be zero
33 int8_t interlace;
34 };
35
36 /*
37 * =====================================================================================
38 * Class: Image
39 * Description: Basic Image Library for PNG and Parrallel Programming Class
40 * =====================================================================================
41 */
42 class Image
43 {
44 public:
45 /* ==================== LIFECYCLE ======================================= */
46 Image (); /* constructor */
47 Image(const char *);
48
49
50 /* ==================== ACCESSORS ======================================= */
51
52 /*
53 *--------------------------------------------------------------------------------------
54 * Class: Image
55 * Method: get_index
56 *--------------------------------------------------------------------------------------
57 */
58 char * get_index ( )
59 {
60 return index;
61 } /* ----- end of method Image::get_index ----- */
62
63 /*
64 *--------------------------------------------------------------------------------------
65 * Class: Image
66 * Method: set_index
67 *--------------------------------------------------------------------------------------
68 */
69 inline void set_index ( char * value )
70 {
71 index = value;
72 return ;
73 } /* ----- end of method Image::set_index ----- */
74
75 /*
76 *--------------------------------------------------------------------------------------
77 * Class: Image
78 * Method: get_pic
79 *--------------------------------------------------------------------------------------
80 */
81 char * get_pic ( )
82 {
83 return pic;
84 } /* ----- end of method Image::get_pic ----- */
85
86 /*
87 *--------------------------------------------------------------------------------------
88 * Class: Image
89 * Method: get_header
90 *--------------------------------------------------------------------------------------
91 */
92 char* get_header()
93 {
94 return loc_header_cpy;
95 } /* ----- end of method Image::get_header ----- */
96
97 /*
98 *--------------------------------------------------------------------------------------
99 * Class: Image
100 * Method: set_header
101 *--------------------------------------------------------------------------------------
102 */
103 void set_header ()
104 {
105 char * start = loc_header_cpy;
106 //loc_header_cpy is an array int he constructor of 8 chars
107 // char * value = get_pic();
108 std::cout << "setting the header array**" << std::endl;
109 std::cout << "**************************" << std::endl;
110 for(int i = 0; i<8; i++){
111 std::cout << "setting the header" << *get_index() << std::endl;
112 *start = *get_index(); //this is a copy
113 std::cout << "header set to =>>" << *start<< std::endl;
114 next_index();
115 start++;
116 }
117 return ;
118 } /* ----- end of method Image::set_header ----- */
119
120
121 /*
122 *--------------------------------------------------------------------------------------
123 * Class: Image
124 * Method: set_pic
125 *--------------------------------------------------------------------------------------
126 */
127 void set_pic ( char * value )
128 {
129 pic = value;
130 return ;
131 } /* ----- end of method Image::set_pic ----- */
132
133 /*
134 *--------------------------------------------------------------------------------------
135 * Class: Image
136 * Method: get_length
137 *--------------------------------------------------------------------------------------
138 */
139 long unsigned int get_length ( )
140 {
141 return length;
142 } /* ----- end of method Image::get_length ----- */
143
144 /*
145 *--------------------------------------------------------------------------------------
146 * Class: Image
147 * Method: set_length
148 *--------------------------------------------------------------------------------------
149 */
150 long unsigned set_length ( long unsigned int value )
151 {
152 length = value;
153 return value ;
154 } /* ----- end of method Image::set_length ----- */
155
156 /*
157 *--------------------------------------------------------------------------------------
158 * Class: Image
159 * Method: get_type
160 *--------------------------------------------------------------------------------------
161 */
162 inline int32_t get_type ( )
163 {
164 return type;
165 } /* ----- end of method Image::get_type ----- */
166
167 /*
168 *--------------------------------------------------------------------------------------
169 * Class: Image
170 * Method: set_type
171 *--------------------------------------------------------------------------------------
172 */
173 int32_t set_type ( int32_t value )
174 {
175 type = value;
176 return value;
177 } /* ----- end of method Image::set_type ----- */
178
179
180 /*
181 *--------------------------------------------------------------------------------------
182 * Class: Image
183 * Method: get_data
184 *--------------------------------------------------------------------------------------
185 */
186 uint8_t * get_data ( )
187 {
188 return data;
189 } /* ----- end of method Image::get_data ----- */
190
191 /*
192 *--------------------------------------------------------------------------------------
193 * Class: Image
194 * Method: set_data
195 *--------------------------------------------------------------------------------------
196 */
197 void set_data ( uint8_t * value )
198 {
199 if(data != nullptr){
200 std::cout << __LINE__ << " We are deleting data" << std::endl;
201 uint8_t * tmp = data;
202 data = nullptr;
203 delete[] tmp;
204 }
205 // std::cout << __LINE__ << " Hello World" << std::endl;
206 std::cout << __LINE__ << " ASSINGING DATA FIELD TO OBJECT" << std::endl;
207 data = value;//the size is determined in get_length()
208 return ;
209 } /* ----- end of method Image::set_data ----- */
210
211 /*
212 *--------------------------------------------------------------------------------------
213 * Class: Image
214 * Method: get_crc
215 *--------------------------------------------------------------------------------------
216 */
217 int32_t get_crc ( )
218 {
219 return crc;
220 } /* ----- end of method Image<T>::get_crc ----- */
221
222 /*
223 *--------------------------------------------------------------------------------------
224 * Class: Image
225 * Method: set_crc
226 *--------------------------------------------------------------------------------------
227 */
228 void set_crc ( int32_t value )
229 {
230 crc = value;
231 return ;
232 } /* ----- end of method Image<T>::set_crc ----- */
233
234 char * get_end(){
235 return end;
236 };
237 void set_end(char * value){
238 end = value;
239 }
240
241
242 /* ==================== MUTATORS ======================================= */
243 void read_png( const char * in = nullptr );
244 void read_header();
245 void read_chunk();
246 void next_index(){
247 char * tmp = get_index();
248 set_index(++tmp);
249 };
250 void header(){
251 read_header();
252 };
253 const char * getNext( );
254
255 /* ==================== OPERATORS ======================================= */
256
257 /* ==================== DATA MEMBERS ======================================= */
258 protected:
259
260 private:
261 char * pic;
262 char loc_header_cpy[8];
263 char * index;
264 char * end;
265 long unsigned int length;
266 uint8_t * data = nullptr;
267 int32_t type;
268 int32_t crc;
269 }; /* ----- end of class Image ----- */
270
271
272 }
273 #endif
Here we can see the constructor opens the file, and loads it into memory, records the size of the file, which is used to end loops later (although one can search of IEND type chunks) and indexes is kept central to the object. For the proposes of the threading, this central indexing is a problem. Each thread will need to take an index and copy it in order to read the data chunks that it requires.
Other information that is stored and reported is chunk length, the CRC for the chunk currently being looked at (again this will need to be localized for the threading), and the header, which is image wide. The implementation of the library files is as follows from png_proj.cpp
1 /*
2 * =====================================================================================
3 *
4 * Filename: png_proj.cpp
5 *
6 * Description: PNG Project
7 *
8 * Version: 1.0
9 * Created: 11/15/2016 12:08:44 PM
10 * Revision: none
11 * Compiler: gcc
12 *
13 * Author: Ruben Safir (mn), ruben@mrbrklyn.com
14 * Company: NYLXS Inc
15 *
16 * =====================================================================================
17 */
18 #include <iostream>
19 #include <thread>
20 #include <zlib.h>
21 #include <iostream>
22 #include <map>
23 #include <string>
24 #include <chrono>
25 #include <thread>
26 #include <mutex>
27 #include <fstream>
28 #include <netinet/in.h>
29 #include "png_proj.h"
30
31 namespace png_proj{
32
33 Image::Image(const char * in)
34 {
35 if(in != nullptr)
36 {
37 this->read_png(in);
38 }
39 }
40
41 void Image::read_png( const char * in)
42 {
43 std::streampos size;
44 std::ifstream file;
45 char * image;
46 if(in == nullptr)
47 {
48 file.open("/home/ruben/png_project/airplane2.PNG", std::ios::in|std::ios::binary|std::ios::ate);
49
50 }else
51 {
52 file.open(in, std::ios::in|std::ios::binary|std::ios::ate);
53 }
54
55 if (file.is_open())
56 {
57 size = file.tellg();
58 image = new char [size];
59 file.seekg (0, std::ios::beg);
60 file.read (image, size);
61 file.close();
62
63 ¦ std::cout << std::endl << "the entire file content is in memory";
64 ¦ std::cout << std::endl << "File Size: " << size << std::endl ;
65 set_length(size);
66 set_index(image);//set the pic buffer index
67 ¦ set_pic(image); //store the pic buffer
68 set_end( get_index() + get_length() );
69 set_header(); //set the internal copy of the header
70 ¦}
71 else{
72 ¦std::cout << std::endl << "Unable to open file";
73 }
74 }
75
76 void Image::read_header()
77 {
78 std::cout << "OK lets read the header!!" << std::endl;
79 if(get_header() == nullptr)
80 {
81 std::cerr << "The header hasn't been initialized yet" << std::endl;
82 //this shouldn't be possible because it is defined as 8 char int in the const
83 //and then it is filled user set_header
84 return;
85 }
86 for(int i = 0;i<8;i++)
87 {
88 std::cout << "index: " << i << " value: " << *( get_header() + i ) << std::endl;
89 }
90 std::cout << std::endl << "done reading header" << std::endl << std::endl << std::endl;
91 }
92
93 void Image::read_chunk()
94 {
95 char * cur = get_index();
96 CHUNK * tmp = reinterpret_cast<CHUNK *>(cur);
97 std::cout << std::endl << "Size of initial Chunk " << sizeof(*tmp) << std::endl;
98 for(int j = 0; j<4; j++){
99 std::cout << std::endl << "==> Raw Chunk Number " << j << ":\tvalue of => " << *tmp << std::endl;
100 switch ( j ) {
101 case 0:
102 std::cout << "***LENGTH****" << std::endl;
103 set_length( ntohl( static_cast<uint32_t>(*tmp) ) );
104 std::cout << "Length value => " << get_length() << std::endl;
105 break;
106 case 1:
107 std::cout << "***TYPE****" << std::endl;
108 set_type(static_cast<int32_t>(*tmp));
109 break;
110 case 2:
111 {
112 std::cout << "***DATA****" << std::endl;
113 unsigned long int l = static_cast<unsigned long int>(get_length());
114 std::cout << "buffer size should be " << get_length() << std::endl;
115 uint8_t * buffer = new uint8_t[l];
116 uint8_t * curarry = reinterpret_cast<uint8_t *>(tmp);
117 std::cout << "buffer element size is " << sizeof(*buffer) << std::endl;
118 std::cout << "buffer size is " << l << std::endl;
119 for(unsigned int k = 0; k < l; k++){
120 std::cout << std::endl << "BYTE NUMBER " << k << std::endl << "********" << std::endl;
121 buffer[k] = curarry[k];
122 std::cout << "data " << buffer[k] << std::endl;
123 std::cout << "tmp " << curarry[k] << std::endl;
124 }
125 set_data(buffer);
126 //cur = ( reinterpret_cast<char*>(tmp) );
127 cur = cur + l;
128 set_index(cur);
129 }
130 break;
131 case 3:
132 std::cout << "***CRC****" << std::endl;
133 set_crc(static_cast<int32_t>(*tmp));
134 break;
135 default:
136 std::cout << "***NOMANDSLAND****" << std::endl;
137 break;
138 } /* ----- end switch ----- */
139
140 if(j != 2){
141 //char * tmp2 = reinterpret_cast<char *>(tmp); //reading each byte
142 std::cout << std::endl << "Out of switch \tSizeof Byte " << sizeof(*cur) << std::endl;
143 for(int i=0; i<4; i++){
144 std::cout << "Character " << i << "::" << std::endl << "\t" << *cur << std::endl;
145 std::cout << "Byte " << i << "::" << std::endl << "\t" << static_cast<unsigned long int>(*cur) << std::endl;
146 cur++;
147 }
148 std::cout<<std::endl;
149 std::cout<<std::endl;
150 //tmp++;
151 //cur = ( reinterpret_cast<char*>(tmp) );
152 set_index(cur);
153 }
154 }
155 set_index(cur);
156 }
157
158 /*
159 ¦*--------------------------------------------------------------------------------------
160 ¦* Class: Image
161 ¦* Method: getNext
162 ¦* Description: gets the next chunk and reads the data
163 ¦*--------------------------------------------------------------------------------------
164 ¦*/
165
166 ¦const char * Image::getNext ()
167 {
168 std::cout << std::endl << "getNext" << std::endl;
169 // next_index();
170 read_chunk();
171 const char *next = get_index();
172 return next;
173 } /* ----- end of method Image::getNext ----- */
174
175
176 }
177
The main action is within the switch. Since we have always for sections per chunk, we can defined needed behaviors for each segment. The length is first read and converted to the correct endian format (since it is an uint32_t type) and read.
The TYPE chunk is just read and reported.
The CRC is just stored for now since it doesn’t affect this project.
The data is then read according to the number of bytes in length from the first segment. This involves BYTE 2 in the loop
110 case 2:
111 {
112 std::cout << "***DATA****" << std::endl;
113 unsigned long int l = static_cast<unsigned long int>(get_length());
114 std::cout << "buffer size should be " << get_length() << std::endl;
115 uint8_t * buffer = new uint8_t[l];
116 uint8_t * curarry = reinterpret_cast<uint8_t *>(tmp);
117 std::cout << "buffer element size is " << sizeof(*buffer) << std::endl;
118 std::cout << "buffer size is " << l << std::endl;
119 for(unsigned int k = 0; k < l; k++){
120 std::cout << std::endl << "BYTE NUMBER " << k << std::endl << "********" << std::endl;
121 buffer[k] = curarry[k];
122 std::cout << "data " << buffer[k] << std::endl;
123 std::cout << "tmp " << curarry[k] << std::endl;
124 }
125 set_data(buffer);
126 //cur = ( reinterpret_cast<char*>(tmp) );
127 cur = cur + l;
128 set_index(cur);
129 }
130 break;
You see here that we recast the data feed according to the type that is needed for our program. It is critical for this to be correct for the reading of data and for the pointer arithmetic to work correctly. This is the reason for my printing out the size of the values of the objects that the pointers are pointing to. It lets me visually see that our pointers are casted correctly.
The output gives us a good look at the main pieces that we are going to target for threading, that is the breakdown of the treads:
1
2 the entire file content is in memory
3 File Size: 19365631
4 setting the header array**
5 **************************
6 setting the header<89>
7 header set to =>><89>
8 setting the headerP
9 header set to =>>P
10 setting the headerN
11 header set to =>>N
12 setting the headerG
13 header set to =>>G
14 setting the header^M
15 header set to =>>^M
16 setting the header
17
18 header set to =>>
19
20 setting the header^Z
21 header set to =>>^Z
22 setting the header
23
24 header set to =>>
25
26 MAIN: line:28 char * image size==>8
27 OK lets read the header!!
28 index: 0 value: <89>
29 index: 1 value: P
30 index: 2 value: N
31 index: 3 value: G
32 index: 4 value: ^M
33 index: 5 value:
34
35 index: 6 value: ^Z
36 index: 7 value:
37
38
39 done reading header
40
41 40
41
42
43 Size of initial Chunk 4
44
45 ==> Raw Chunk Number 0: value of => 218103808
46 ***LENGTH****
47 Length value => 13
48
49 Out of switch Sizeof Byte 1
50 Character 0::
51 ^@
52 Byte 0::
53 0
54 Character 1::
55 ^@
56 Byte 1::
57 0
58 Character 2::
59 ^@
60 Byte 2::
61 0
62 Character 3::
63 ^M
64 Byte 3::
65 13
66
67
68
69 ==> Raw Chunk Number 1: value of => 218103808
70 ***TYPE****
71
72 Out of switch Sizeof Byte 1
73 Character 0::
74 I
75 Byte 0::
76 73
77 Character 1::
78 H
79 Byte 1::
80 72
81 Character 2::
82 D
83 Byte 2::
84 68
85 Character 3::
86 R
87 Byte 3::
88 82
89
90
91
92 ==> Raw Chunk Number 2: value of => 218103808
93 ***DATA****
94 buffer size should be 13
95 buffer element size is 1
96 buffer size is 13
97
98 BYTE NUMBER 0
99 ********
100 data ^@
101 tmp ^@
102
103 BYTE NUMBER 1
104 ********
105 data ^@
106 tmp ^@
and so on
158 BYTE NUMBER 12
159 ********
160 data ^@
161 tmp ^@
162 206 ASSINGING DATA FIELD TO OBJECT
163
164 ==> Raw Chunk Number 3: value of => 218103808
165 ***CRC****
166
167 Out of switch Sizeof Byte 1
168 Character 0::
169 <86>
170 Byte 0::
171 18446744073709551494
172 Character 1::
173 -
174 Byte 1::
175 45
176 Character 2::
177 <
178 Byte 2::
179 60
180 Character 3::
181 Þ
182 Byte 3::
183 18446744073709551582
184
185
186
187 getNext
After the CRC field, we have the whole header. The header data contains information about this file that we can use later (like the reported size of the file). But we skip this and then head straight for the inner part of the PNG graphics
the next chunk is of the non-critical type gama:
189 Size of initial Chunk 4
190
191 ==> Raw Chunk Number 0: value of => 67108864
192 ***LENGTH****
193 Length value => 4
194
195 Out of switch Sizeof Byte 1
196 Character 0::
197 ^@
198 Byte 0::
199 0
200 Character 1::
201 ^@
202 Byte 1::
203 0
204 Character 2::
205 ^@
206 Byte 2::
207 0
208 Character 3::
209 ^D
210 Byte 3::
211 4
212
213
214
215 ==> Raw Chunk Number 1: value of => 67108864
216 ***TYPE****
217
218 Out of switch Sizeof Byte 1
219 Character 0::
220 g
221 Byte 0::
222 103
223 Character 1::
224 A
225 Byte 1::
226 65
227 Character 2::
228 M
229 Byte 2::
230 77
231 Character 3::
232 A
233 Byte 3::
234 65
Moving on the next the non-critical public type bKGD
287
288 Hey I am not Kosher Bird, yoy can't eat me!!
289
290 getNext
291
292 Size of initial Chunk 4
293
294 ==> Raw Chunk Number 0: value of => 100663296
295 ***LENGTH****
296 Length value => 6
297
298 Out of switch Sizeof Byte 1
299 Character 0::
300 ^@
301 Byte 0::
302 0
303 Character 1::
304 ^@
305 Byte 1::
306 0
307 Character 2::
308 ^@
309 Byte 2::
310 0
311 Character 3::
312 ^F
313 Byte 3::
314 6
315
316
317
318 ==> Raw Chunk Number 1: value of => 100663296
319 ***TYPE****
320
321 Out of switch Sizeof Byte 1
322 Character 0::
323 b
324 Byte 0::
325 98
326 Character 1::
327 K
328 Byte 1::
329 75
330 Character 2::
331 G
332 Byte 2::
333 71
334 Character 3::
335 D
336 Byte 3::
337 68
338
339
then comes the critical chunk type PHYs – which I will also pretty much ignore for this project
401 Hey I am not Kosher Bird, yoy can't eat me!!
402
403 getNext
404
405 Size of initial Chunk 4
406
407 ==> Raw Chunk Number 0: value of => 150994944
408 ***LENGTH****
409 Length value => 9
410
411 Out of switch Sizeof Byte 1
412 Character 0::
413 ^@
414 Byte 0::
415 0
416 Character 1::
417 ^@
418 Byte 1::
419 0
420 Character 2::
421 ^@
422 Byte 2::
423 0
424 Character 3::
425
426 Byte 3::
427 9
428
429
430
431 ==> Raw Chunk Number 1: value of => 150994944
432 ***TYPE****
433
434 Out of switch Sizeof Byte 1
435 Character 0::
436 p
437 Byte 0::
438 112
439 Character 1::
440 H
441 Byte 1::
442 72
443 Character 2::
444 Y
445 Byte 2::
446 89
447 Character 3::
448 s
449 Byte 3::
450 115
451
and another one, which we will bypass to reach a time stamp chunk
529 Hey I am not Kosher Bird, yoy can't eat me!!
530
531 getNext
532
533 Size of initial Chunk 4
534
535 ==> Raw Chunk Number 0: value of => 117440512
536 ***LENGTH****
537 Length value => 7
538
539 Out of switch Sizeof Byte 1
540 Character 0::
541 ^@
542 Byte 0::
543 0
544 Character 1::
545 ^@
546 Byte 1::
547 0
548 Character 2::
549 ^@
550 Byte 2::
551 0
552 Character 3::
553 ^G
554 Byte 3::
555 7
556
557
558
559 ==> Raw Chunk Number 1: value of => 117440512
560 ***TYPE****
561
562 Out of switch Sizeof Byte 1
563 Character 0::
564 t
565 Byte 0::
566 116
567 Character 1::
568 I
569 Byte 1::
570 73
571 Character 2::
572 M
573 Byte 2::
574 77
575 Character 3::
576 E
577 Byte 3::
578 69
579
until we finally reach the array of IDAT chunks which have lengths of 8192 bytes each
651 Size of initial Chunk 4
652
653 ==> Raw Chunk Number 0: value of => 2097152
654 ***LENGTH****
655 Length value => 8192
656
657 Out of switch Sizeof Byte 1
658 Character 0::
659 ^@
660 Byte 0::
661 0
662 Character 1::
663 ^@
664 Byte 1::
665 0
666 Character 2::
667
668 Byte 2::
669 32
670 Character 3::
671 ^@
672 Byte 3::
673 0
674
675
676
677 ==> Raw Chunk Number 1: value of => 2097152
678 ***TYPE****
679
680 Out of switch Sizeof Byte 1
681 Character 0::
682 I
683 Byte 0::
684 73
685 Character 1::
686 D
687 Byte 1::
688 68
689 Character 2::
690 A
691 Byte 2::
692 65
693 Character 3::
694 T
695 Byte 3::
696 84
697
698
699
700 ==> Raw Chunk Number 2: value of => 2097152
701 ***DATA****
702 buffer size should be 8192
703 buffer element size is 1
704 buffer size is 8192
705
706 BYTE NUMBER 0
707 ********
708 data ^@
709 tmp ^@
710
711 BYTE NUMBER 1
712 ********
713 data ^@
714 tmp ^@
715
…
41737 BYTE NUMBER 8191
41738 ********
41739 data ÿ
41740 tmp ÿ
41741 200 We are deleting data
41742 206 ASSINGING DATA FIELD TO OBJECT
41743
41744 ==> Raw Chunk Number 3: value of => 2097152
41745 ***CRC****
41746
41747 Out of switch Sizeof Byte 1
41748 Character 0::
41749 <8c>
41750 Byte 0::
41751 18446744073709551500
41752 Character 1::
41753 s
41754 Byte 1::
41755 115
41756 Character 2::
41757 ^Q
41758 Byte 2::
41759 17
41760 Character 3::
41761
41762 Byte 3::
41763 18446744073709551533
41764
41765
and so on through the rest of the file until we reach the end
41766 Hey I am not Kosher Bird, yoy can't eat me!!
41767
41768 getNext
41769
41770 Size of initial Chunk 4
41771
41772 ==> Raw Chunk Number 0: value of => 2097152
41773 ***LENGTH****
41774 Length value => 8192
41775
41776 Out of switch Sizeof Byte 1
41777 Character 0::
41778 ^@
41779 Byte 0::
41780 0
41781 Character 1::
41782 ^@
41783 Byte 1::
41784 0
41785 Character 2::
41786
41787 Byte 2::
41788 32
41789 Character 3::
41790 ^@
41791 Byte 3::
41792 0
41793
41794
41795
41796 ==> Raw Chunk Number 1: value of => 2097152
41797 ***TYPE****
41798
41799 Out of switch Sizeof Byte 1
41800 Character 0::
41801 I
41802 Byte 0::
41803 73
41804 Character 1::
41805 D
41806 Byte 1::
41807 68
41808 Character 2::
41809 A
41810 Byte 2::
41811 65
41812 Character 3::
41813 T
41814 Byte 3::
41815 84
41816
41817
41818
41819 ==> Raw Chunk Number 2: value of => 2097152
41820 ***DATA****
41821 buffer size should be 8192
41822 buffer element size is 1
41823 buffer size is 8192
41824
41825 BYTE NUMBER 0
41826 ********
41827 data ^@
41828 tmp ^@
41829
41830 BYTE NUMBER 1
41831 ********
41832 data ^@
41833 tmp ^@
41834
and on
97028124 Hey I am not Kosher Bird, yoy can't eat me!!
97028125
97028126 getNext
97028127
97028128 Size of initial Chunk 4
97028129
97028130 ==> Raw Chunk Number 0: value of => 0
97028131 ***LENGTH****
97028132 Length value => 0
97028133
97028134 Out of switch Sizeof Byte 1
97028135 Character 0::
97028136 ^@
97028137 Byte 0::
97028138 0
97028139 Character 1::
97028140 ^@
97028141 Byte 1::
97028142 0
97028143 Character 2::
97028144 ^@
97028145 Byte 2::
97028146 0
97028147 Character 3::
97028148 ^@
97028149 Byte 3::
97028150 0
97028151
97028152
97028153
97028154 ==> Raw Chunk Number 1: value of => 0
97028155 ***TYPE****
97028156
97028157 Out of switch Sizeof Byte 1
97028158 Character 0::
97028159 I
97028160 Byte 0::
97028161 73
97028162 Character 1::
97028163 E
97028164 Byte 1::
97028165 69
97028166 Character 2::
97028167 N
97028168 Byte 2::
97028169 78
97028170 Character 3::
97028171 D
97028172 Byte 3::
97028173 68
97028174
97028175
97028176
97028177 ==> Raw Chunk Number 2: value of => 0
97028178 ***DATA****
97028179 buffer size should be 0
97028180 buffer element size is 1
97028181 buffer size is 0
97028182 200 We are deleting data
97028183 206 ASSINGING DATA FIELD TO OBJECT
97028184
97028185 ==> Raw Chunk Number 3: value of => 0
97028186 ***CRC****
97028187
97028188 Out of switch Sizeof Byte 1
97028189 Character 0::
97028190 ®
97028191 Byte 0::
97028192 18446744073709551534
97028193 Character 1::
97028194 B
97028195 Byte 1::
97028196 66
97028197 Character 2::
97028198 `
97028199 Byte 2::
97028200 96
97028201 Character 3::
97028202 <82>
97028203 Byte 3::
97028204 18446744073709551490
97028205
97028206
DONE
Clearly, we have an opportunity to parallize the chunk reading. What are the factors that we need to account for when we paralize the attach on reading the PNG file
A) File Location Tracking
Each chunk is in order and as we separate the processing from the order then we will need to reassemble the pieces in order to maintain a coherent image.
B) Localization of indexing
Indexing is currently being done on the object level. While that still needs to happen so that chunks can be parsed out correctly, the byte size indexing through the data needs to be localized to the chunk. This is likely best done in C++ by leveraging encapsulation by making
new chunk type objects which have their own indexes and which are called by threads.
C) Localization of Data
Currently chunk data is stored in the image object. When a chunks data is aquired, the old data is deleted off the heap and the new data is passed to the object. We will need the data to remain intact with the new chunk object until it is assembled in the image object to its destination, may that be a file or to a GPU.
D) Decompression moved to Chunks
E) setting an optimal number of threads.
This is all that I wanted for this project, but other opportunities are available for parallelization
A) PNG can be hooked directly into opengl and cuda. In fact, such hooks are build into libpng
B) Compression itself can be done parallel if the size of a chunk is large enough to justify it.
C) Geometric design can be added along with interlacing. By physically dividing the image into subdividing blocks, recursion can be applied and binary trees can be applied to it. This is especially true when creating PNG files.