Skip to content

Comments

add custom encoding registration function#7

Merged
thda merged 1 commit intothda:masterfrom
wiro34:feature/register_custom_encoding
Mar 29, 2019
Merged

add custom encoding registration function#7
thda merged 1 commit intothda:masterfrom
wiro34:feature/register_custom_encoding

Conversation

@wiro34
Copy link
Contributor

@wiro34 wiro34 commented Mar 28, 2019

I'm new to golang.

This is experimental.

I hope somebody that advise if there is better way.

Fixes #6

Usage Sample:

package foo

import (
	"github.com/axgle/mahonia"
	"golang.org/x/text/encoding"
	"golang.org/x/text/encoding/japanese"
	"golang.org/x/text/transform"
)

var ShiftJIS encoding.Encoding = &sjisEncoding{}

type sjisEncoding struct {
	encoding.Encoding
}

func (e *sjisEncoding) NewDecoder() *encoding.Decoder {
	return japanese.ShiftJIS.NewDecoder()
}

func (e *sjisEncoding) NewEncoder() *encoding.Encoder {
	return &encoding.Encoder{Transformer: &shiftJISEncoder{}}
}

type shiftJISEncoder struct{ transform.NopResetter }

// Transform using mahonia.
func (shiftJISEncoder) Transform(dst, src []byte, atEOF bool) (nDst, nSrc int, err error) {
	encoder := mahonia.NewEncoder("sjis")
	encoded := []byte(encoder.ConvertString(string(src)))
	copy(dst, encoded)
	return nDst + len(encoded), nSrc + len(src), err
}
// override `sjis` encoding
tds.RegisterEncoding("sjis", foo.ShiftJIS)

@thda
Copy link
Owner

thda commented Mar 28, 2019

Hi,

many thanks for your input.

I strongly suggest using utf-8 for your use case. That said, if your server does not support it, maybe it would be easier to add a flag to disable charset conversion, named CharConvert, which could be set to off.
This flag would be taken into account in session.go, in the processEnvChange function.
It would probably give better performance.

That said, I'm not opposed to registering a charset encoding as you have done.
Before I merge this, would you mind adding a read lock to getEncoding, and a read/write lock to register encoding?
Would fix eventual race conditions.

A quick word about this in the documentation (doc.go) would also be appreciated.

Thanks,
Thomas

@wiro34 wiro34 force-pushed the feature/register_custom_encoding branch from 8df5bdb to 4a84fdf Compare March 29, 2019 09:23
@wiro34
Copy link
Contributor Author

wiro34 commented Mar 29, 2019

I strongly suggest using utf-8 for your use case.

Yes, I know I should use utf8. However, our server is still not...

I updated the commit.

Thank you for your advice.
I tried s.charConvert = false, but it didn't work.
How is charConvert used?

@thda
Copy link
Owner

thda commented Mar 29, 2019

S.charconvert is not used. I'll add that connection flag and make use of it.
Let me review and merge today. Thanks again!

@thda
Copy link
Owner

thda commented Mar 29, 2019

Merging this. I'll add the charConvert connexion parameter and run all my tests before pushing a release.

@thda thda merged commit f6d0943 into thda:master Mar 29, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants