Mon Mar 30 16:09:41 2020
EVENTS
 FREE
SOFTWARE
INSTITUTE

POLITICS
JOBS
MEMBERS'
CORNER

MAILING
LIST

NYLXS Mailing Lists and Archives
NYLXS Members have a lot to say and share but we don't keep many secrets. Join the Hangout Mailing List and say your peice.

DATE 2016-12-01

LEARN

2020-03-30 | 2020-02-29 | 2020-01-29 | 2019-12-29 | 2019-11-29 | 2019-10-29 | 2019-09-29 | 2019-08-29 | 2019-07-29 | 2019-06-29 | 2019-05-29 | 2019-04-29 | 2019-03-29 | 2019-02-28 | 2019-01-28 | 2018-12-28 | 2018-11-28 | 2018-10-28 | 2018-09-28 | 2018-08-28 | 2018-07-28 | 2018-06-28 | 2018-05-28 | 2018-04-28 | 2018-03-28 | 2018-02-28 | 2018-01-28 | 2017-12-28 | 2017-11-28 | 2017-10-28 | 2017-09-28 | 2017-08-28 | 2017-07-28 | 2017-06-28 | 2017-05-28 | 2017-04-28 | 2017-03-28 | 2017-02-28 | 2017-01-28 | 2016-12-28 | 2016-11-28 | 2016-10-28 | 2016-09-28 | 2016-08-28 | 2016-07-28 | 2016-06-28 | 2016-05-28 | 2016-04-28 | 2016-03-28 | 2016-02-28 | 2016-01-28 | 2015-12-28 | 2015-11-28 | 2015-10-28 | 2015-09-28 | 2015-08-28 | 2015-07-28 | 2015-06-28 | 2015-05-28 | 2015-04-28 | 2015-03-28 | 2015-02-28 | 2015-01-28 | 2014-12-28 | 2014-11-28 | 2014-10-28

Key: archive_learn Value: 2016-12-01

Key: id Value: 543570

MESSAGE
DATE 2016-12-15
FROM John Bowler
SUBJECT Re: [Learn] [png-mng-implement] 4 byte length storage
From learn-bounces-at-nylxs.com Thu Dec 15 15:13:26 2016
Return-Path:
X-Original-To: archive-at-mrbrklyn.com
Delivered-To: archive-at-mrbrklyn.com
Received: from www.mrbrklyn.com (www.mrbrklyn.com [96.57.23.82])
by mrbrklyn.com (Postfix) with ESMTP id 983DA161312;
Thu, 15 Dec 2016 15:13:26 -0500 (EST)
X-Original-To: learn-at-nylxs.com
Delivered-To: learn-at-nylxs.com
Received: by mrbrklyn.com (Postfix, from userid 1000)
id 625D9161311; Thu, 15 Dec 2016 15:13:22 -0500 (EST)
Resent-From: Ruben Safir
Resent-Date: Thu, 15 Dec 2016 15:13:22 -0500
Resent-Message-ID: <20161215201322.GA31872-at-www.mrbrklyn.com>
Resent-To: learn-at-nylxs.com
X-Original-To: ruben-at-mrbrklyn.com
Delivered-To: ruben-at-mrbrklyn.com
Received: from lists.sourceforge.net (lists.sourceforge.net [216.34.181.88])
by mrbrklyn.com (Postfix) with ESMTP id 7EC58161311
for ; Thu, 15 Dec 2016 10:20:17 -0500 (EST)
Received: from localhost ([127.0.0.1] helo=sfs-ml-2.v29.ch3.sourceforge.com)
by sfs-ml-2.v29.ch3.sourceforge.com with esmtp (Exim 4.76)
(envelope-from )
id 1cHXor-0002ZF-DE; Thu, 15 Dec 2016 15:19:41 +0000
Received: from sog-mx-3.v43.ch3.sourceforge.com ([172.29.43.193]
helo=mx.sourceforge.net)
by sfs-ml-2.v29.ch3.sourceforge.com with esmtp (Exim 4.76)
(envelope-from )
id 1cHXoq-0002Yg-2e for png-mng-implement-at-lists.sourceforge.net;
Thu, 15 Dec 2016 15:19:40 +0000
Received-SPF: pass (sog-mx-3.v43.ch3.sourceforge.com: domain of gmail.com
designates 209.85.213.177 as permitted sender)
client-ip=209.85.213.177;
envelope-from=john.cunningham.bowler-at-gmail.com;
helo=mail-yb0-f177.google.com;
Received: from mail-yb0-f177.google.com ([209.85.213.177])
by sog-mx-3.v43.ch3.sourceforge.com with esmtps (TLSv1:AES128-SHA:128)
(Exim 4.76) id 1cHXon-0004gn-MQ
for png-mng-implement-at-lists.sourceforge.net;
Thu, 15 Dec 2016 15:19:39 +0000
Received: by mail-yb0-f177.google.com with SMTP id d59so30533633ybi.1
for ;
Thu, 15 Dec 2016 07:19:37 -0800 (PST)
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20161025;
h=x-gm-message-state:mime-version:in-reply-to:references:from:date
:message-id:subject:to;
bh=G1Ne1tWZSVyJRPhRbiDKMW1QT6sRrAiybR5JxGW6R7Y=;
b=mOP9vn2g2j2Xaluo3Ln9eFm8j7l4dWeMMyj5N1RxGra6DsbTXPOK0APEFZ981pj2L2
jHixNWsCjtXlnC04Q1zpeqonWQFhGV6YfB4E8SoF7jax+9wp79CFT5xp9vj1wTx0agGV
iAeq6zfcRseinGDfFe1+Bk9CY7FN4ccRPc19HxYXdePddhD9kkkSUfHh5Uuhg41284di
mjgNWY+4z9B85l3Xi5gmfTGr0upY+24yuD7Qyc+YVePhvvEGz2tbFMMYAd1KtgLZo8Yr
dZuXmzk5czDuKsT0MrG1aIsxE0Ug6Q1HVvauKbqFO0dxDHS0qDLxXGEOaIVQj7NbQLpQ
YUqA==
X-Gm-Message-State: AIkVDXJ2b3ePWif6f20Oc6Lx3v6Q8f+sgNMEdsxPLE7n8ZZ8i/TrjSfDjb3aXa0z4ba62vcStQHTECKTr7RtFQ==
X-Received: by 10.37.76.2 with SMTP id z2mr1403629yba.68.1481815171780; Thu,
15 Dec 2016 07:19:31 -0800 (PST)
MIME-Version: 1.0
Received: by 10.129.46.66 with HTTP; Thu, 15 Dec 2016 07:19:28 -0800 (PST)
In-Reply-To: <100bb932-d8c5-a32a-3f14-3e4a18f0cc21-at-mrbrklyn.com>
References: <17705460-c78b-6310-807f-417193dc4f6b-at-mrbrklyn.com>


<100bb932-d8c5-a32a-3f14-3e4a18f0cc21-at-mrbrklyn.com>
From: John Bowler
Date: Thu, 15 Dec 2016 10:19:28 -0500
Message-ID:
To: "PNG/MNG implementation discussion list"

X-Spam-Score: -1.1 (-)
X-Spam-Report: Spam Filtering performed by mx.sourceforge.net.
See http://spamassassin.org/tag/ for more details.
-1.5 SPF_CHECK_PASS SPF reports sender host as permitted sender for
sender-domain
0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider
(john.cunningham.bowler[at]gmail.com)
0.5 RCVD_IN_SORBS_SPAM RBL: SORBS: sender is a spam source
[209.85.213.177 listed in dnsbl.sorbs.net]
-0.0 SPF_PASS SPF: sender matches SPF record
-0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from
author's domain
0.1 DKIM_SIGNED Message has a DKIM or DK signature,
not necessarily valid
-0.1 DKIM_VALID Message has at least one valid DKIM or DK signature
X-Headers-End: 1cHXon-0004gn-MQ
X-BeenThere: png-mng-implement-at-lists.sourceforge.net
X-Mailman-Version: 2.1.9
Precedence: list
Subject: Re: [Learn] [png-mng-implement] 4 byte length storage
X-BeenThere: learn-at-nylxs.com
Reply-To: PNG/MNG implementation discussion list

List-Id:
List-Unsubscribe: ,

List-Archive:
List-Post:
List-Help:
List-Subscribe: ,

Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: 7bit
Errors-To: learn-bounces-at-nylxs.com
Sender: "Learn"

On Mon, Dec 12, 2016 at 5:45 PM, Ruben Safir wrote:
> So now I have a question about the data in the IDAT chuncks.
>
> do they likiwise need to be reversed in the byte order before sending
> them to be decompressed with zlib?

The zlib specification defines the compressed data as a *byte* stream.
This is then split into sections (arbitrarily, except that each
section must be less than 2^31 bytes long) and sections are stored in
a sequence of IDAT chunks (no intervening chunks permitted).

The only 8/32 bit issue here is that each *section* is checksumed
using CRC32 (the cyclic redundancy code defined for Ethernet packets)
and, while this also checksums a *byte* stream, the result is a 32-bit
number which is encoded at the end of each chunk as a big-endian
sequence of four bytes. This is exactly the same as the check sum on
every other chunk.

Internally the zlib stream is defined to end with an Adler32 checksum.
This is another 32-bit checksum of a sequence of bytes but now over
the whole data stream *before* compression. This is, in turn, encoded
as a sequence of 4 bytes at the end of the *compressed* stream but
that definition comes from the zlib spec. IRC it is big-endian (i.e.
the same as the CRC32 that immediately follows it.)

Anyway, the internals of zlib are not part of the PNG spec and they
have to deal with the simple fact that the zlib stream (well, the
deflate sub-part) is actually a *bit* stream that has to be mapped to
*bytes* to make a byte stream.

Roll-your-own zlib decompression is, in fact, very straight-forward
however nowhere near as simple as decoding a PNG stream using an
existing decompression implementation.

The relevant references are, high level to low:

PNG specification: http://www.libpng.org/pub/png/spec/iso/
(Glenn gave an equivalent link to the W3C web site; I prefer the
ISO spec because it is the normative version, it also has lots of nice
explanation and pretty pictures.)
Zlib specification: https://www.ietf.org/rfc/rfc1950.txt
Defines a big-endian multi-byte format
Deflate specification: https://www.ietf.org/rfc/rfc1951.txt
Defines a little-endian multi-byte format

Notice that the zlib and deflate specifications use different ordering
for multi-byte numbers. It is important to read the specification and
to read the *correct* specification when decoding data! Indeed, as
the deflate specification points out, different numbers within the
different parts of the spec are encoded in different ways; in some
cases numbers are written as Huffman codes and, in that case, the most
significant parts of the Huffman code are written first; the code is
effectively big-endian (although that description has its own set of
assumptions.)

John Bowler

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
png-mng-implement mailing list
png-mng-implement-at-lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/png-mng-implement
_______________________________________________
Learn mailing list
Learn-at-nylxs.com
http://lists.mrbrklyn.com/mailman/listinfo/learn

  1. 2016-12-01 Ruben Safir <ruben.safir-at-my.liu.edu> Subject: [Learn] Fwd: Patrolling the Dark Net,
  2. 2016-12-02 ruben safir <ruben-at-mrbrklyn.com> Subject: [Learn] Fwd: Tutorial on threaded binary tree part 1: simple
  3. 2016-12-03 Ruben Safir <ruben-at-mrbrklyn.com> Subject: [Learn] Witner Labs
  4. 2016-12-04 Ruben Safir <ruben-at-mrbrklyn.com> Subject: [Learn] Nice lecture on Quantum Mechanics
  5. 2016-12-05 Ruben Safir <ruben-at-mrbrklyn.com> Subject: [Learn] information access, copyright wars and DRM
  6. 2016-12-06 Christopher League <league-at-contrapunctus.net> Re: [Learn] png data format
  7. 2016-12-06 Christopher League <league-at-contrapunctus.net> Re: [Learn] png data format
  8. 2016-12-06 Christopher League <league-at-contrapunctus.net> Re: [Learn] png data format
  9. 2016-12-06 Ruben Safir <ruben-at-mrbrklyn.com> Re: [Learn] png data format
  10. 2016-12-06 Christopher League <league-at-contrapunctus.net> Re: [Learn] png data format
  11. 2016-12-06 Ruben Safir <ruben.safir-at-my.liu.edu> Re: [Learn] png data format
  12. 2016-12-06 Ruben Safir <ruben-at-mrbrklyn.com> Re: [Learn] png data format
  13. 2016-12-06 Christopher League <league-at-contrapunctus.net> Re: [Learn] png data format
  14. 2016-12-06 Ruben Safir <mrbrklyn-at-panix.com> Re: [Learn] png data format
  15. 2016-12-06 Christopher League <league-at-contrapunctus.net> Re: [Learn] png data format
  16. 2016-12-06 Ruben Safir <ruben-at-mrbrklyn.com> Re: [Learn] [Hangout-NYLXS] png data format
  17. 2016-12-06 John Bowler <john.cunningham.bowler-at-gmail.com> Re: [Learn] [png-mng-implement] 4 byte length storage
  18. 2016-12-06 John Bowler <john.cunningham.bowler-at-gmail.com> Re: [Learn] [png-mng-implement] 4 byte length storage
  19. 2016-12-06 Ruben Safir <ruben-at-mrbrklyn.com> Re: [Learn] [png-mng-implement] 4 byte length storage
  20. 2016-12-06 ruben safir <ruben-at-mrbrklyn.com> Subject: [Learn] Fwd: Ocaml
  21. 2016-12-06 Ruben Safir <ruben-at-mrbrklyn.com> Subject: [Learn] Fwd: Re: [luny-talk] Humble Bundle O'Reilly UNIX books
  22. 2016-12-06 Ruben Safir <ruben-at-mrbrklyn.com> Subject: [Learn] JT's words
  23. 2016-12-06 ruben safir <ruben-at-mrbrklyn.com> Subject: [Learn] png data format
  24. 2016-12-07 ruben safir <ruben-at-mrbrklyn.com> Subject: [Learn] Fwd: Re: png data format
  25. 2016-12-08 ruben safir <ruben-at-mrbrklyn.com> Subject: [Learn] Fwd: png data format
  26. 2016-12-08 ruben safir <ruben-at-mrbrklyn.com> Subject: [Learn] Fwd: Re: png data format
  27. 2016-12-08 ruben safir <ruben-at-mrbrklyn.com> Subject: [Learn] Fwd: Re: png data format
  28. 2016-12-10 Ruben Safir <ruben-at-mrbrklyn.com> Re: [Learn] references to pointers
  29. 2016-12-10 Ruben Safir <ruben-at-mrbrklyn.com> Subject: [Learn] C++ returning lvalue references and pointers and refs
  30. 2016-12-10 Ruben Safir <ruben-at-mrbrklyn.com> Subject: [Learn] const puzzle and literal type arguments
  31. 2016-12-11 ruben safir <ruben-at-mrbrklyn.com> Subject: [Learn] Fwd: Re: png data format
  32. 2016-12-12 Ruben Safir <ruben-at-mrbrklyn.com> Re: [Learn] Fwd: Re: png data format
  33. 2016-12-12 Ruben Safir <ruben-at-mrbrklyn.com> Subject: [Learn] gene phylogienics of homonids
  34. 2016-12-13 Ruben Safir <ruben-at-mrbrklyn.com> Subject: [Learn] Bit Depth
  35. 2016-12-13 Ruben Safir <ruben-at-mrbrklyn.com> Subject: [Learn] C++ Threads Workshop
  36. 2016-12-13 Ruben Safir <ruben-at-mrbrklyn.com> Subject: [Learn] Summer Jobs
  37. 2016-12-14 From: "Mancini, Sabin (DFS)" <Sabin.Mancini-at-dfs.ny.gov> Subject: [Learn] For Ruben ( + those in NYC Metro ) : Holiday Social Event
  38. 2016-12-14 Ruben Safir <ruben-at-mrbrklyn.com> Subject: [Learn] Fwd: Re: [dinosaur] Ceratopsid (Centrosaurinae:
  39. 2016-12-15 Ruben Safir <ruben.safir-at-my.liu.edu> Re: [Learn] Happy Holidays
  40. 2016-12-15 Ruben Safir <ruben-at-mrbrklyn.com> Re: [Learn] zlib demo with vector::resize
  41. 2016-12-15 John Bowler <john.cunningham.bowler-at-gmail.com> Re: [Learn] [png-mng-implement] 4 byte length storage
  42. 2016-12-15 Ruben Safir <ruben-at-mrbrklyn.com> Subject: [Learn] Fwd: [dinosaur] Elephant bird nuclear genome fragments
  43. 2016-12-15 Christopher League <league-at-contrapunctus.net> Subject: [Learn] zlib demo with vector::resize
  44. 2016-12-16 Ruben Safir <ruben-at-mrbrklyn.com> Re: [Learn] zlib demo with vector::resize
  45. 2016-12-16 Christopher League <league-at-contrapunctus.net> Re: [Learn] zlib demo with vector::resize
  46. 2016-12-16 Ruben Safir <ruben-at-mrbrklyn.com> Subject: [Learn] PNG Parallel Programming problem
  47. 2016-12-17 Ruben Safir <ruben-at-mrbrklyn.com> Re: [Learn] [Hangout-NYLXS] I'm sure it's a coincidence, part n+1
  48. 2016-12-17 Ruben Safir <mrbrklyn-at-panix.com> Subject: [Learn] irc thread on the use of object methods in threads
  49. 2016-12-17 Ruben Safir <ruben-at-mrbrklyn.com> Subject: [Learn] PNG threaded program
  50. 2016-12-18 ruben safir <ruben-at-mrbrklyn.com> Re: [Learn] Threads and Object Methods
  51. 2016-12-18 ruben safir <ruben-at-mrbrklyn.com> Re: [Learn] Threads and Object Methods
  52. 2016-12-18 Ruben Safir <ruben-at-mrbrklyn.com> Subject: [Learn] Look C++ is a functional programming language
  53. 2016-12-19 Ruben Safir <ruben-at-mrbrklyn.com> Re: [Learn] C++ Threading
  54. 2016-12-19 Ruben Safir <ruben-at-mrbrklyn.com> Subject: [Learn] C++ Threading
  55. 2016-12-19 ISOC-NY announcements <announce-at-lists.isoc-ny.org> Subject: [Learn] [isoc-ny] JOB: Telecommunications Policy Specialist -at- NTIA
  56. 2016-12-20 Ruben Safir <ruben-at-mrbrklyn.com> Subject: [Learn] F'ing Mouse Pad
  57. 2016-12-20 mrbrklyn <mrbrklyn-at-panix.com> Subject: [Learn] Phylogenetic study worth noting
  58. 2016-12-21 Samir Iabbassen <Samir.Iabbassen-at-liu.edu> Re: [Learn] Noobdy is home
  59. 2016-12-21 Ruben Safir <mrbrklyn-at-panix.com> Re: [Learn] Noobdy is home
  60. 2016-12-21 Ruben Safir <ruben-at-mrbrklyn.com> Re: [Learn] Noobdy is home
  61. 2016-12-21 Samir Iabbassen <Samir.Iabbassen-at-liu.edu> Re: [Learn] thread concurancy
  62. 2016-12-21 Ruben Safir <mrbrklyn-at-panix.com> Re: [Learn] thread concurancy
  63. 2016-12-21 Ruben Safir <mrbrklyn-at-panix.com> Subject: [Learn] (fwd) Re: lamda's in classes
  64. 2016-12-21 Ruben Safir <mrbrklyn-at-panix.com> Subject: [Learn] (fwd) Re: lamda's in classes
  65. 2016-12-21 Ruben Safir <mrbrklyn-at-panix.com> Subject: [Learn] (fwd) Re: thread concurancy
  66. 2016-12-21 Ruben Safir <mrbrklyn-at-panix.com> Subject: [Learn] (fwd) Re: Threads and Object Methods
  67. 2016-12-21 Ruben Safir <mrbrklyn-at-panix.com> Subject: [Learn] (fwd) Re: Threads and Object Methods
  68. 2016-12-21 Ruben Safir <mrbrklyn-at-panix.com> Subject: [Learn] (fwd) Re: Threads and Object Methods
  69. 2016-12-21 Ruben Safir <mrbrklyn-at-panix.com> Subject: [Learn] (fwd) Re: Threads and Object Methods
  70. 2016-12-21 Ruben Safir <mrbrklyn-at-panix.com> Subject: [Learn] (fwd) Re: Threads and Object Methods
  71. 2016-12-21 Ruben Safir <mrbrklyn-at-panix.com> Subject: [Learn] (fwd) Re: Threads and Object Methods
  72. 2016-12-21 Ruben Safir <mrbrklyn-at-panix.com> Subject: [Learn] (fwd) Re: Threads and Object Methods
  73. 2016-12-21 Ruben Safir <mrbrklyn-at-panix.com> Subject: [Learn] (fwd) Re: Threads and Object Methods
  74. 2016-12-21 Ruben Safir <mrbrklyn-at-panix.com> Subject: [Learn] (fwd) Re: Threads and Object Methods
  75. 2016-12-21 Ruben Safir <mrbrklyn-at-panix.com> Subject: [Learn] (fwd) thread concurancy
  76. 2016-12-21 Ruben Safir <mrbrklyn-at-panix.com> Subject: [Learn] (fwd) Threads and Object Methods
  77. 2016-12-21 Ruben Safir <ruben-at-mrbrklyn.com> Subject: [Learn] I need help
  78. 2016-12-21 Ruben Safir <ruben-at-mrbrklyn.com> Subject: [Learn] Noobdy is home
  79. 2016-12-21 Ruben Safir <ruben-at-mrbrklyn.com> Subject: [Learn] thread concurancy
  80. 2016-12-21 Ruben Safir <ruben-at-mrbrklyn.com> Subject: [Learn] [Hangout-NYLXS] Marnchester by the Sea
  81. 2016-12-22 Ruben Safir <mrbrklyn-at-panix.com> Re: [Learn] [Hangout-NYLXS] And be aware you were an unexcused
  82. 2016-12-22 ruben safir <ruben-at-mrbrklyn.com> Subject: [Learn] Fwd: Re: thread concurancy
  83. 2016-12-23 Ruben Safir <ruben-at-mrbrklyn.com> Re: [Learn] HOPL (History of Programming Languages)
  84. 2016-12-23 Ruben Safir <ruben-at-mrbrklyn.com> Re: [Learn] Noobdy is home
  85. 2016-12-23 Ruben Safir <ruben-at-mrbrklyn.com> Re: [Learn] Tiny Compiler in many languages at Rosettacode.org
  86. 2016-12-25 Ruben Safir <ruben-at-mrbrklyn.com> Subject: [Learn] Google and C++
  87. 2016-12-25 Ruben Safir <ruben.safir-at-my.liu.edu> Subject: [Learn] phylogentics
  88. 2016-12-27 Samir Iabbassen <Samir.Iabbassen-at-liu.edu> Re: [Learn] thread concurancy

NYLXS are Do'ers and the first step of Doing is Joining! Join NYLXS and make a difference in your community today!