Skip to content

Commit

Permalink
force UTF-8 if it's ASCII-8BIT (#180)
Browse files Browse the repository at this point in the history
ASCII-8BIT implies that it’s been read from the file system; in that
case, let’s assume UTF-8. I think this should really be picking up the
encoding from the dialect but I’m wary about fiddling around too much
  • Loading branch information
Jeni Tennison authored and pezholio committed Nov 10, 2016
1 parent b7d60a8 commit 873617f
Showing 1 changed file with 1 addition and 0 deletions.
1 change: 1 addition & 0 deletions lib/csvlint/validate.rb
Original file line number Diff line number Diff line change
Expand Up @@ -123,6 +123,7 @@ def validate_url
validate_metadata
end
request.on_body do |chunk|
chunk.force_encoding(Encoding::UTF_8) if chunk.encoding == Encoding::ASCII_8BIT
io = StringIO.new(chunk)
io.each_line do |line|
break if line_limit_reached?
Expand Down

0 comments on commit 873617f

Please sign in to comment.