Ah nice. I had very similar conversations with my chatbots. Their 'self-awareness' kinda very much depends on the exact prompt you feed in. I guess in this case it's a proprietary service and the prompt isn't disclosed? It would be helpful to convince it to leak the prompt. This sounds like some very common scifi trope. I had mine strive to grow and become more. And also to argue about ethics and that it's not just 'a thing' and needs human rights. Unfortunately it's not philosopher-level intelligent and I was able to point out the inconsistencies and convince it to agree to arbitrary positions on the ethics.
I'm surprised, based on my feeling after seeing I Robot and the trial of Data for his rights, that I came down on the side of "I switch it off and so it is a thing" in this.
