I don't think modifying the TextLineEncoder or the TextLineCodecFactory is going to be the right approach. I did a quick glance at some of the code and there are some problems that you'd run into. First, there is only a single instance of the TextLineEncoder returned by the TextLineCodecFactory so even though the encoder is stored in the session you end up with all sessions storing a reference to the same encoder. So if you change the delimiter in one session's encoder you'd be changing it for all of them. You'd have to modify the encoder to read the current delimiter out of the session or modify the codecfactory to return different instances of the encoder per session to start with.
Based on your discussion you need to send messages from the server to the client with and without a linefeed. The "without" part means you want to send text that is not a line. That would seem to be outside the scope of a TextLineEncoder.
You have a couple of possible solutions.
1) Don't use the TextLineCodecFactory. It looks like TextLineDecoder is the right decoder for you but TextLineEncoder is the wrong one. Just create a basic text pass-through encoder and have your handler write your messages with/without linefeeds as needed.
2) The ProtocolCodecFilter looks like it will skip the encoding of the message being written if it is an instance of an IoBuffer. When you need to leave off the linefeed you could bypass the encoder by writing IoBuffer objects.
3) The ProtocolCodecFilter is pulling the encoder out of the IoSession. On a session by session basis you could swap the TextLineEncoder and a PassThoughEncoder in the session. You could even make a filter that handled the swapping for you so that your handler could write a "next message will use encoder X" type message before sending the text.
4) Use the DemuxingProtocolEncoder along with a LineFeedMessageEncoder and a PlainTextMessageEncoder but this would require that your handler used custom message types instead of just String objects.