torch engine support chatglm3

[grimoire]

ChatGLM3 shares most codes with ChatGLM2.

RMSNorm has been replaced with our triton kernel.

Fix batched inference.

Caution

I am not able to add model template in lmdeploy/model.py since the tokenizer can not encode special token to the special ids.

   def encode(self, s: str, bos: bool = False, eos: bool = False) -> List[int]:
       assert type(s) is str
       t = self.sp_model.encode(s)
       if bos:
           t = [self.bos_id] + t
       if eos:
           t = t + [self.eos_id]
       return t

   def decode(self, t: List[int]) -> str:
       text, buffer = "", []
       for token in t:
           if token in self.index_special_tokens:

               if buffer:
                   text += self.sp_model.decode(buffer)
                   buffer = []
               text += self.index_special_tokens[token]
           else:
               buffer.append(token)
       if buffer:
           text += self.sp_model.decode(buffer)
       return text

[lvhan028]

May remove the chat template in model.py if it doesn't work

[lvhan028]

LGTM

[lvhan028]

The chat template of chatGLM models is still an issue. @AllentDan may follow it up.

[AllentDan]

The chat template of chatGLM models is still an issue. @AllentDan may follow it up.

Encoding may not be the hardest part. It also influences incremental decoding.