Time Parsing fix for #EXT-X-PROGRAM-DATE-TIME #78

md2k · 2017-01-31T15:15:54Z

While HLS Draft in section 3.4.5 require that time should comply with RFC3339, during my work i noticed that many Encoders use ISO8601 where Timezone representation is less strict than RFC3339.
ISO8601 allow to expose TZ in several ways:

DateTime/Zone
<time>Z
<time>±hh:mm (by RFC3339)
<time>±hhmm
<time>±hh

Example of manifest where we getting Parser error:

#EXTM3U
#EXT-X-TARGETDURATION:8
#EXT-X-PROGRAM-DATE-TIME:2017-01-30T17:26:04+0100
#EXT-X-MEDIA-SEQUENCE:855733
#EXTINF:8,
1478947334-fast-1920-1080-2750000-855733.ts
#EXTINF:8,
1478947334-fast-1920-1080-2750000-855734.ts
#EXTINF:8,
1478947334-fast-1920-1080-2750000-855735.ts

Error itself:
parsing time "2017-01-30T17:26:04+0100" as "2006-01-02T15:04:05.999999999Z07:00": cannot parse "+0100" as "Z07:00"

But while RFC3339 newer then ISO8601, lot of software still use it, so my proposal is to create Format validation for incoming tag #EXT-X-PROGRAM-DATE-TIME in reader.go and using switch{case} test incoming string against precompiled regexp rules, and route string to time.Parse() with proper layout(format).

I implemented only RFC3339 with Nano seconds, this should be enough and in case we need add other formats this will be easy to add to switch{case} chain.

coveralls · 2017-01-31T15:19:05Z

Coverage increased (+0.5%) to 59.482% when pulling d05b0b3 on md2k:datetime_fix into 644e009 on grafov:master.

md2k · 2017-01-31T15:22:01Z

Can see it failed against Go 1.4.3 and 1.5.4 because oldest Go time lib do not support 2 digits Time zone offset (+-00).
I can remove it if we want to keep old GoLang support as well.

grafov · 2017-01-31T21:47:26Z

I not have statistics how many encoders don't respect RFC rules for time values but I hope it is lesser than number of properly defined encoders. So I think it is not good add complexity to the parsing function by default. Anyway it hard to count any possible variations of time formats. But the problem seems the real case so it maybe solution:

var TimeParse func(value string) (time.Time, error) = DefaultTimeParse

func DefaultTimeParse(value string) (time.Time, error) {
	return time.Parse(time.RFC3339Nano, value)
}

// ...

func decodeLineOfMediaPlaylist(p *MediaPlaylist, wv *WV, state *decodingState, l
// ...
  	case !state.tagProgramDateTime && strings.HasPrefix(line, "#EXT-X-PROGRAM-DATE-TIME:"):
  		state.tagProgramDateTime = true
  		state.listType = MEDIA
		if state.programDateTime, err = TimeParse(line[25:]); strict && err != nil {
			return err
// ...
}

It offers default parser for RFC3339 but allow you customize it for any nonstandard formats you will find in playlists. So if you want universal parser you can bring complex logic with switch{} into you own function and just pass it before parsing as m3u8.TimeParse = myTimeParser.

It will not change API of existing functions. Only one drawback I see it is global parser for a whole application. It maybe solved with interface{} based solution but it is a more verbose and I not sure we need such complexity for this task.

md2k · 2017-02-01T08:33:50Z

We can try use your approach for me personally it is same if i will write my own parser or it will be inside library. because i need do this check all the time due unavailability of knowledge what encoder will send to me :)
if we speak about how many encoders use old fashion, i not think too often, i have personally few suppliers with in total of 6-8 streams like this (it not big amount to compare with what we have in total, but we not able to mitigate this on supplier side) . Also usage of this Tag in manifests over my statistic is pretty rare in general.

But keep in mind that ISO8601 is absolutely valid format, and formally ISO is above RFC if we speak about some real-life certifications. My opinion that GoLang gurus should take care about formats inside time library and respect ISO8601/RFC3339 and not only last one (like this done in many other languages), so maybe it not so bad idea to implement in-place small complexity.

Any way, i'll wait for your last word, and then we can finally go to in-place detection or as your approach with external parser.

md2k · 2017-02-01T09:37:01Z

Also if we stick with in-place detection, we can remove this format <time>±hh i not think this really will be used at all, in my practice i never met such layout.
https://golang.org/src/time/format.go?s=22597:22643#L34
Strange that they mention about timezone formats but in reality their Layout do not respect this without manual changes in layout string we passing into the Parse function.
They seems simply offloading this to programmers :(
https://golang.org/src/time/format.go?s=22597:22643#L131

md2k · 2017-02-01T09:59:31Z

Benchmark for large playlist without Time detection:

BenchmarkDecodeMasterPlaylist-8   	   10000	    132447 ns/op
BenchmarkDecodeMediaPlaylist-8    	     100	  13764477 ns/op
BenchmarkEncodeMasterPlaylist-8   	 2000000	       872 ns/op
BenchmarkEncodeMediaPlaylist-8    	     500	   3793081 ns/op

with time detection:

BenchmarkDecodeMasterPlaylist-8   	   10000	    134999 ns/op
BenchmarkDecodeMediaPlaylist-8    	     100	  13827311 ns/op
BenchmarkEncodeMasterPlaylist-8   	 2000000	       859 ns/op
BenchmarkEncodeMediaPlaylist-8    	     500	   3815003 ns/op

bradleyfalzon · 2017-02-01T10:00:40Z

Personally, if @grafov wants to support auto detecting of the format, I'd consider something like: https://play.golang.org/p/JRR0vboAke

Thanks for running benchmarks too, can you run it with https://godoc.org/golang.org/x/tools/cmd/benchcmp and run tests with -count 20 or something similar?

md2k · 2017-02-01T10:02:37Z

Will do, also for your suggestion.

grafov · 2017-02-01T10:31:54Z

In the evening I'll do benchmarks for using wrapping function. Though it is more question of code clarity than speed. Personally I'd like the way with wrapping function than direct comparing of multiple formats in the code (it is no matter they realized with switch{} or with for{}).

I think the default realization covers the most of cases. If you need cover rare case you add custom code. Though the library can offer two ready for use funcs: strict parser for a single format (as used now) and complex parser for a bunch of time formats (for example with code as @bradleyfalzon presented). User can select one of them by assign to TimeParse variable or provide his own parser in a function.

md2k · 2017-02-01T10:57:57Z

@grafov actually this is not matter if this is rare case or not, it more question of support complete ISO standard or just part of it, because in GoLang you should take care about such support on your own, while in Python,C,Java and cetra this done by time library of those languages.
We not speak about other standards, but only about ISO8601, and RFC3339 is simply implements this ISO by throwing away some parts of it. in short:
ISO allow instead of T use just space bar, in fact i never seen that anyone did that, but it part of standard. there few more things which is allowed by ISO , but not allowed by RFC.
In fact we should not allow use any other formats, because HLS Draft utilize ISO8601/RFC3339. so i think we simple create wrapper for time parser which detects for us ISO8601 in most common variants inside the library instead as external "callback" style function.

for example HLS Draft state that it use ISO8601: https://tools.ietf.org/html/draft-pantos-http-live-streaming-20#section-4.3.2.6

The date/time representation is ISO/IEC 8601:2004 [ISO_8601] and
   SHOULD indicate a time zone and fractional parts of seconds

In their example they show that timezone use colons, but in fact ISO8601:2004 (oldest as well) allow you 3(even4) types of TZ representations (UTZ as only 'Z' added, just ZHH, ZHHMM and in some cases ZHHMMSS), where Z can be also + or - depends we speak about positive value of UTC offset or negative offset.
check section 4.2.5.2 Local time and the difference from UTC and 4.2.5 Local time and Coordinated Universal Time (UTC)
Link to full ISO http://www.uai.cl/images/sitio/biblioteca/citas/ISO_8601_2004en.pdf

Here is bench for my switch{case} and for(loop) implementations: https://gist.github.com/md2k/ebd0a2d8c51d372dadbfcdcb51905fc5

I did them on MacOS i7 3.4GHZ per core (4 real Cores), 16GB RAM, SSD backed HDD
(Not Have linux OS at this moment to do benches there, can do this in few hours )

md2k · 2017-02-01T11:49:11Z

I think previous bench not accurate, did new one on Linux Ubuntu LTS 14.04, CPU info in gist as well.

Bench: https://gist.github.com/md2k/778beb3185ac181857a7da63f02d63d6

Looks like forloop will be faster, expectable :)
And for me not clear why BenchmarkDecodeMasterPlaylist, BenchmarkEncodeMasterPlaylist and BenchmarkEncodeMediaPlaylist have differences, because our functionality should affect only BenchmarkDecodeMediaPlaylist

Added GOGC=800 for ForLoop variant (Original stays the same, just to show performance where GC is a bit more relaxed), ~+12-17% boost for Decoding operations:
https://gist.github.com/md2k/778beb3185ac181857a7da63f02d63d6

P.S. No offence by the way :) excuse me if some of my sentence looks a bit rude

md2k · 2017-02-01T13:01:58Z

Also added to Bench gist from my previous message Regexp Vs For-loop

bradleyfalzon · 2017-02-01T13:20:29Z

My apologies @md2k I was thinking of the tool https://godoc.org/rsc.io/benchstat to allow better comparisons. Sorry, I had gotten them confused.

md2k · 2017-02-01T13:25:08Z

@bradleyfalzon oh oh.. but even with old one it clearly visible that more or less they equal.

coveralls · 2017-02-01T13:26:04Z

Coverage increased (+0.7%) to 59.706% when pulling 3849d84 on md2k:datetime_fix into 644e009 on grafov:master.

md2k · 2017-02-01T13:55:37Z

@bradleyfalzon

-count 20

Original VS Regexp

name                    old time/op  new time/op  delta
DecodeMasterPlaylist-8  20.3µs ± 7%  20.4µs ± 9%    ~     (p=0.822 n=18+19)
DecodeMediaPlaylist-8   15.4ms ± 2%  15.6ms ± 8%    ~     (p=0.612 n=16+19)
EncodeMasterPlaylist-8   822ns ± 1%   818ns ± 1%  -0.42%  (p=0.000 n=19+18)
EncodeMediaPlaylist-8   3.66ms ± 2%  3.68ms ± 2%    ~     (p=0.080 n=19+19)

Original Vs Loop

name                    old time/op  new time/op  delta
DecodeMasterPlaylist-8  20.3µs ± 7%  20.4µs ± 4%    ~     (p=0.799 n=18+19)
DecodeMediaPlaylist-8   15.4ms ± 2%  15.7ms ± 6%  +1.82%  (p=0.015 n=16+19)
EncodeMasterPlaylist-8   822ns ± 1%   823ns ± 1%    ~     (p=0.241 n=19+20)
EncodeMediaPlaylist-8   3.66ms ± 2%  3.66ms ± 1%    ~     (p=0.212 n=19+19)

Regexp Vs Loop

name                    old time/op  new time/op  delta
DecodeMasterPlaylist-8  20.4µs ± 9%  20.4µs ± 4%    ~     (p=0.624 n=19+19)
DecodeMediaPlaylist-8   15.6ms ± 8%  15.7ms ± 6%    ~     (p=0.339 n=19+19)
EncodeMasterPlaylist-8   818ns ± 1%   823ns ± 1%  +0.60%  (p=0.000 n=18+20)
EncodeMediaPlaylist-8   3.68ms ± 2%  3.66ms ± 1%    ~     (p=0.091 n=19+19)

md2k · 2017-02-03T16:43:33Z

Hi @grafov didn't had any chance to look into this yet?

grafov · 2017-02-04T14:43:01Z

Sorry for delay @md2k. I carefully reread all you notes and HLS Draft about datetime parsing. You are right, really HLS Draft follows ISO/IEC 8601:2004 but only recommends (because of "SHOULD" word) "indicate a time zone and fractional parts of seconds". So any valid format from ISO 8601:2004 may be passed to playlists and restriction of our library with RFC3339 is not correct. Thank you for pointing to this problem. Well it looks reasonable support formats of ISO 8601:2004 in the library by default. Let's do it.

I looked at you benchmarks results and also did my benchmarks too just for compare how various implementations will affect parsing. I tested not whole decoders from the package but only fragments with datetime parsing. Code of benchmark is here: https://play.golang.org/p/Ti981TW5cC

My results are:

BenchmarkTimePackage-8       	 2000000	       747 ns/op
BenchmarkTimeFuncDefault-8   	 2000000	       759 ns/op
BenchmarkTimeFuncFull-8      	 2000000	       742 ns/op
BenchmarkTimeSwitchRegex-8   	  500000	      2480 ns/op
BenchmarkTimeLoop-8          	 2000000	       773 ns/op

So it looks loop realization is fastest and only slightly affects performance. I think we don't need regexps for mathching rows firstly because time.Parse functions fast enough. I also think we SHOULD provide way for customization with TimeParse function that may be reassigned by the library user. It will allow pass more nonstandard formats and vice versa narrow parsing to only single format if the user wants more strict playlist validation. So I propose two functions:

var TimeParse func(format, value string) (time.Time, error) = FullTimeParse

// Just wrapper around time.Parse()
func StrictTimeParse(format, value string) (time.Time, error) {
	return time.Parse(time.RFC3339Nano, value)
}

// Parse several formats.
func FullTimeParse(layout, value string) (time.Time, error) {
        layouts := []string{
	"2006-01-02T15:04:05.999999999Z0700",
	"2006-01-02T15:04:05.999999999Z07:00",
	"2006-01-02T15:04:05.999999999Z07",
	}
	var (
		err error
		t   time.Time
	)
	for _, layout := range layouts {
		if t, err = time.Parse(layout, value); err == nil {
			return t, nil
		}
	}
	return t, err
}

By default we will use FullTimeParse so the decoder will parse various formats without errors. But if you need more strict validation or just return the old behaviour you can assign StrictTimeParse.

md2k · 2017-02-06T09:33:48Z

Hi @grafov, That is fine. i did tests with long playlist by adding there required TAG, so this maybe why it not shows a big difference in benchmarks. did 2 branches locally with regexp and loop and tested them by calling BenchmarkDecodeMediaPlaylist().

And i'm fine with you proposal.

md2k · 2017-02-07T10:49:38Z

Hi @grafov, do you want me to amend my change to apply your proposal for this pull request ?

grafov · 2017-02-07T11:58:30Z

Git amend will be ok, I not see problems with it. We can also close the pull request and create a new one. I have no preferences here, use a way more comfortable for you.

md2k · 2017-02-22T08:10:34Z

Pardon for being a silent for a while. lot of work. next week i have some free window, so will push update to merge.

grafov · 2017-02-22T11:57:05Z

Sure no problems, thank you for notify!

coveralls · 2017-03-03T11:37:24Z

Coverage increased (+1.3%) to 60.294% when pulling 03d4770 on md2k:datetime_fix into f55aa7a on grafov:master.

md2k · 2017-03-03T11:37:49Z

@grafov i pushed merge (my branch in sync with your latest master commits),
only what is worrying me , it is that go 1.4.x and 1.5.x do not support ISO completely, they fixed it only since 1.6.x :( 2 digits timzone will fail for 1.4 and 1.5 (but sure i not think that some one will pass such information to playlists, but it affects Travis tests)

bradleyfalzon

This is good, a few comments throughout, but the implementation looks good now.

As for the 1.4.x and 1.5.x, I'd personally be fine with dropping that support @grafov.

bradleyfalzon · 2017-03-04T02:08:21Z

reader.go

@@ -638,3 +646,27 @@ func decodeLineOfMediaPlaylist(p *MediaPlaylist, wv *WV, state *decodingState, l
 	}
 	return err
 }
+
+// Strict Time Wrapper implements RFC3339 with Nanoseconds accuracy


Should be // StrictTimeParse implements.... with a trailing period.

bradleyfalzon · 2017-03-04T02:08:51Z

reader.go

+	return time.Parse(DATETIME, value)
+}
+
+// Custom Time Parser implements ISO/IEC 8601:2004


Should be // FileTimeParse implements... with a trailing period.

bradleyfalzon · 2017-03-04T02:10:35Z

reader.go

 			return err
 		}
+


I'm unsure why there's new lines here, can these be removed?

This is not me, it is my formatter in Sublime Go :)
but sure i can remove it through console editor

bradleyfalzon · 2017-03-04T02:15:03Z

reader_test.go

+// We testing ISO/IEC 8601:2004 where we can get time in UTC, UTC with Nanoseconds
+// timeZone in formats '±00:00', '±0000', '±00'
+// m3u8.FullTimeParse()
+func TestTimeLayoutsDecode(t *testing.T) {


This test is really just testing the FullTimeParse function, but it's testing it via decodeLineOfMediaPlaylist, I'd just call this function directly.

Also, there's a lot of repetition in the tests, can you rewrite using https://github.com/golang/go/wiki/TableDrivenTests (or sub tests) similar to

m3u8/writer_test.go

Lines 223 to 253 in f55aa7a

// Create new media playlist

// Add segment to media playlist

// Set encryption key

func TestSetKeyForMediaPlaylist(t *testing.T) {

tests := []struct {

KeyFormat string

KeyFormatVersions string

ExpectVersion uint8

}{

{"", "", 3},

{"Format", "", 5},

{"", "Version", 5},

{"Format", "Version", 5},

}

for _, test := range tests {

p, e := NewMediaPlaylist(3, 5)

if e != nil {

t.Fatalf("Create media playlist failed: %s", e)

}

if e = p.Append("test01.ts", 5.0, ""); e != nil {

t.Errorf("Add 1st segment to a media playlist failed: %s", e)

}

if e := p.SetKey("AES-128", "https://example.com", "iv", test.KeyFormat, test.KeyFormatVersions); e != nil {

t.Errorf("Set key to a media playlist failed: %s", e)

}

if p.ver != test.ExpectVersion {

t.Errorf("Set key playlist version: %v, expected: %v", p.ver, test.ExpectVersion)

}

}

}

?

bradleyfalzon · 2017-03-04T02:15:35Z

reader_test.go

+// We testing Strict format of RFC3339 where we can get time in UTC, UTC with Nanoseconds
+// timeZone in formats '±00:00', '±0000', '±00'
+// m3u8.StrictTimeParse()
+func TestStrictTimeLayoutsDecode(t *testing.T) {


This test is really just testing the StrictTimeParse function, but it's testing it via decodeLineOfMediaPlaylist, I'd just call this function directly.

Also, there's a lot of repetition in the tests, can you rewrite using https://github.com/golang/go/wiki/TableDrivenTests (or sub tests) similar to

m3u8/writer_test.go

Lines 223 to 253 in f55aa7a

// Create new media playlist

// Add segment to media playlist

// Set encryption key

func TestSetKeyForMediaPlaylist(t *testing.T) {

tests := []struct {

KeyFormat string

KeyFormatVersions string

ExpectVersion uint8

}{

{"", "", 3},

{"Format", "", 5},

{"", "Version", 5},

{"Format", "Version", 5},

}

for _, test := range tests {

p, e := NewMediaPlaylist(3, 5)

if e != nil {

t.Fatalf("Create media playlist failed: %s", e)

}

if e = p.Append("test01.ts", 5.0, ""); e != nil {

t.Errorf("Add 1st segment to a media playlist failed: %s", e)

}

if e := p.SetKey("AES-128", "https://example.com", "iv", test.KeyFormat, test.KeyFormatVersions); e != nil {

t.Errorf("Set key to a media playlist failed: %s", e)

}

if p.ver != test.ExpectVersion {

t.Errorf("Set key playlist version: %v, expected: %v", p.ver, test.ExpectVersion)

}

}

}

?

ok will check links and re-do tests.

bradleyfalzon · 2017-03-04T02:15:59Z

structure.go

 	DATETIME = time.RFC3339Nano // Format for EXT-X-PROGRAM-DATE-TIME defined in section 3.4.5
+


I'm unsure why there's new lines here, can these be removed?

same as first time, SublimeGo formatter

grafov · 2017-03-04T06:53:31Z

I agree. Support for golang < 1.6 dropped for Travis builds.

coveralls · 2017-03-13T10:41:32Z

Coverage increased (+6.5%) to 65.571% when pulling bffb0b6 on md2k:datetime_fix into f55aa7a on grafov:master.

md2k · 2017-03-13T10:54:38Z

Hi @grafov commit in place.

bradleyfalzon

Looks great, but there's a couple stray newlines that can be removed, I don't think they should be there.

bradleyfalzon · 2017-03-13T21:50:46Z

reader.go

@@ -501,7 +507,8 @@ func decodeLineOfMediaPlaylist(p *MediaPlaylist, wv *WV, state *decodingState, l
 	case !state.tagProgramDateTime && strings.HasPrefix(line, "#EXT-X-PROGRAM-DATE-TIME:"):
 		state.tagProgramDateTime = true
 		state.listType = MEDIA
-		if state.programDateTime, err = time.Parse(DATETIME, line[25:]); strict && err != nil {
+


Can we remove this newline?

bradleyfalzon · 2017-03-13T21:50:57Z

structure.go

@@ -31,7 +31,8 @@ const (
 		   o  The EXT-X-MEDIA tag.
 		   o  The AUDIO and VIDEO attributes of the EXT-X-STREAM-INF tag.
 	*/
-	minver   = uint8(3)
+	minver = uint8(3)
+


Can we remove this newline?

coveralls · 2017-03-14T09:02:27Z

Coverage increased (+6.6%) to 65.631% when pulling 4ce59d5 on md2k:datetime_fix into f55aa7a on grafov:master.

bradleyfalzon · 2017-03-14T09:14:30Z

This looks brilliant to me, @grafov if you're happy, I think it's ready to squash and merge.

bradleyfalzon · 2017-03-14T09:14:53Z

And thanks @md2k for the revisions, it's appreciated.

md2k · 2017-03-14T09:26:19Z

always welcome :)

grafov added the enhancement label Jan 31, 2017

md2k added 4 commits March 3, 2017 12:30

EXT-X-PROGRAM-DATE-TIME Fix

6c81630

test for time pareser part

4f72517

rename variables

cbf3011

Global TimeParse function with custom parsers support

03d4770

md2k force-pushed the datetime_fix branch from 3849d84 to 03d4770 Compare March 3, 2017 11:34

bradleyfalzon suggested changes Mar 4, 2017

View reviewed changes

small fixes to tests and comments

bffb0b6

bradleyfalzon suggested changes Mar 13, 2017

View reviewed changes

removed few newlines

4ce59d5

bradleyfalzon approved these changes Mar 14, 2017

View reviewed changes

grafov merged commit 3cf03b2 into grafov:master Mar 19, 2017

	// Create new media playlist
	// Add segment to media playlist
	// Set encryption key
	func TestSetKeyForMediaPlaylist(t *testing.T) {
	tests := []struct {
	KeyFormat string
	KeyFormatVersions string
	ExpectVersion uint8
	}{
	{"", "", 3},
	{"Format", "", 5},
	{"", "Version", 5},
	{"Format", "Version", 5},
	}

	for _, test := range tests {
	p, e := NewMediaPlaylist(3, 5)
	if e != nil {
	t.Fatalf("Create media playlist failed: %s", e)
	}
	if e = p.Append("test01.ts", 5.0, ""); e != nil {
	t.Errorf("Add 1st segment to a media playlist failed: %s", e)
	}
	if e := p.SetKey("AES-128", "https://example.com", "iv", test.KeyFormat, test.KeyFormatVersions); e != nil {
	t.Errorf("Set key to a media playlist failed: %s", e)
	}
	if p.ver != test.ExpectVersion {
	t.Errorf("Set key playlist version: %v, expected: %v", p.ver, test.ExpectVersion)
	}
	}
	}

		DATETIME = time.RFC3339Nano // Format for EXT-X-PROGRAM-DATE-TIME defined in section 3.4.5

Time Parsing fix for #EXT-X-PROGRAM-DATE-TIME #78

Time Parsing fix for #EXT-X-PROGRAM-DATE-TIME #78

Conversation

md2k commented Jan 31, 2017

coveralls commented Jan 31, 2017 • edited Loading

md2k commented Jan 31, 2017 • edited Loading

grafov commented Jan 31, 2017 • edited Loading

md2k commented Feb 1, 2017

md2k commented Feb 1, 2017 • edited Loading

md2k commented Feb 1, 2017

bradleyfalzon commented Feb 1, 2017 • edited Loading

md2k commented Feb 1, 2017

grafov commented Feb 1, 2017 • edited Loading

md2k commented Feb 1, 2017

md2k commented Feb 1, 2017 • edited Loading

md2k commented Feb 1, 2017

bradleyfalzon commented Feb 1, 2017

md2k commented Feb 1, 2017

coveralls commented Feb 1, 2017 • edited Loading

md2k commented Feb 1, 2017 • edited Loading

md2k commented Feb 3, 2017

grafov commented Feb 4, 2017

md2k commented Feb 6, 2017 • edited Loading

md2k commented Feb 7, 2017

grafov commented Feb 7, 2017

md2k commented Feb 22, 2017

grafov commented Feb 22, 2017

coveralls commented Mar 3, 2017 • edited Loading

md2k commented Mar 3, 2017

bradleyfalzon left a comment • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

grafov commented Mar 4, 2017 • edited Loading

coveralls commented Mar 13, 2017 • edited Loading

md2k commented Mar 13, 2017

bradleyfalzon left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

coveralls commented Mar 14, 2017 • edited Loading

bradleyfalzon commented Mar 14, 2017 • edited Loading

bradleyfalzon commented Mar 14, 2017

md2k commented Mar 14, 2017

coveralls commented Jan 31, 2017 •

edited

Loading

md2k commented Jan 31, 2017 •

edited

Loading

grafov commented Jan 31, 2017 •

edited

Loading

md2k commented Feb 1, 2017 •

edited

Loading

bradleyfalzon commented Feb 1, 2017 •

edited

Loading

grafov commented Feb 1, 2017 •

edited

Loading

md2k commented Feb 1, 2017 •

edited

Loading

coveralls commented Feb 1, 2017 •

edited

Loading

md2k commented Feb 1, 2017 •

edited

Loading

md2k commented Feb 6, 2017 •

edited

Loading

coveralls commented Mar 3, 2017 •

edited

Loading

bradleyfalzon left a comment •

edited

Loading

grafov commented Mar 4, 2017 •

edited

Loading

coveralls commented Mar 13, 2017 •

edited

Loading

coveralls commented Mar 14, 2017 •

edited

Loading

bradleyfalzon commented Mar 14, 2017 •

edited

Loading