The goal of getting people to think of AI as conscious is very subtly driven by a desire to disprove the existence of the soul.
If you can get people to think a very "smart" machine is conscious, then you seemly prove that humans don't have a soul that animates their bodies, mind, and will, making it rather that we are biological machines instead. After all, AI can do and know so much compared to a human, at least intellectually, thus humans don't have a soul and never did.
If humans don't have a soul....well, you get the idea based upon the worst 20th Century implementations of 19th Century ideas.
The word you're looking for is "conscious," meaning "self-aware and experiencing a personal reality," not "conscience," meaning "sense of right and wrong."
We don't know what consciousness is. We truly have no idea. Which means we can't measure whether anything else has achieved consciousness. We don't know for sure whether mice are conscious. We don't know whether consciousness is a binary or a spectrum. We don't know fucking anything about it.
We're looking for an answer, but we're not really looking for a soul, because that seems to be more of an idea than, like, something we can actually interact with.
Ha, dang autocorrect, which is ironic. Thanks.
That said, the soul is the thing that animates living creatures. Things that can't be measured or seen can still exist, and can be shown to exist by the results of their existence.
You sure about that definition? I mean, maybe you'd say a dog has a soul, but - an ant? A sunflower? A bacterium?