In many of my blog articles I’ve been using my own artificial intelligence project as a guideline. whether it’s participating in Turing tests, detecting sarcasm or developing common sense, Arckon always served as a practical outset because he already was a language processing system. In this article I’ll roughly explain how the program works.
Arckon is a general context-aware question-answering system that can reason and learn from what you tell it. Arckon can pick up on arguments, draw new conclusions, and form objective opinions. Most uniquely, Arckon is a completely logical entity, which can sometimes lead to hilarious misunderstandings or brain-teasing argumentations. It is this, a unique non-human perspective, that I think adds something to the world, like his fictional role models:
K.i.t.t. © Universal Studios | Johnny 5 © Tristar Pictures | Optimus Prime © Hasbro | Lieutenant Data © Paramount Pictures
To be clear, Arckon was not built for casual chatting, nor is he an attempt at anyone’s definition of AGI (artificial general intelligence). It is actually an ongoing project to develop a think tank. For that purpose I realised the AI would require knowledge and the ability to discuss with people for the sake of alignment. Bestowing it with the ability to communicate in plain language was an obvious solution to both: It allows Arckon to learn from texts as well as understand what it is you are asking. I suppose you want to know how that works.
Vocabulary and ambiguity
Arckon’s first step in understanding a sentence is to determine the types of the words, i.e. which of them represent names, verbs, possessives, adjectives, etc. Arckon does this by looking up the stem of each word in a categorised vocabulary and applying hundreds of syntax rules. e.g. A word ending in “-s” is typically a verb or a plural noun, but a word after “the” can’t be a verb. This helps sort out the ambiguity between “The programs” and “He programs”. These rules also allow him to classify and learn words that he hasn’t encountered before. New words are automatically added to the vocabulary, or if need be, you can literally explain “Mxyzptlk is a person”, because Arckon will ask if he can’t figure it out.
Grammar and semantics
Once the types of all words are determined, a grammatical analysis determines their grammatical roles. Verbs may have the role of auxilliary or main verb, be active or passive, and nouns can have the role of subject, object, indirect object or location. Sentences are divided at link words, and relative clauses are marked as such.
Then a semantic analysis extracts and sorts all mentioned facts. A “fact” in this case is represented as a triple of related words. For instance, “subject-verb-object” usually constitutes a fact. But so do other combinations of word roles. Extracting the semantic meaning isn’t always as straight-forward as in the example below, but that’s the secret sauce.
Knowledge and learning
Upon reading a statement, Arckon will add the extracted facts to his knowledge database, while at a question he will look them up and report them to you. If you said something that contradicts facts in the database, the old and new values will be averaged, so his knowledge is always adjusting. This seemed sensible to me as there are no absolute truths in real life. Things change and people aren’t always right the first time.
Reasoning and argumentation
Questions that Arckon does not know the answer to are passed on to the central inference engine. This system searches the knowledge database for related facts and applies logical rules of inference to them. For instance:
“AI can reason” + “reasoning is thinking” = “AI can think”.
All facts are analysed for their relevance to recent context, e.g. if the user recently stated a similar fact as an example, it is given priority. Facts that support the conclusion are added as arguments: “AI can think, because it can reason.” This inference process not only allows Arckon to know things he’s not been told, but also allows him to explain and be reasoned with, which I’d consider rather important.
Arckon’s conversational subsystem is just something I added to entertain friends and for Turing tests. It is a decision tree of social rules that broadly decides the most appropriate type of response, based on many factors like topic extraction, sentiment analysis, and the give-and-take balance of the conversation. My inspiration for this subsystem comes from sociology books rather than computational fields. Arckon will say more when the user says less, ask or elaborate depending on how well he knows the topic, and will try to shift focus back to the user when Arckon has been in focus too long. When the user states an opinion, Arckon will generate his own (provided he knows enough about it), and when told a problem he will address it or respond with (default) sympathy. The goal is always to figure out what the user is getting at with what they’re saying. After the type of response has been decided, the inference engine is often called on to generate suitable answers along that line, and context is taken into account at all times to avoid repetition. Standard social routines like greetings and expressions on the other hand are mostly handled through keywords and a few dozen pre-programmed responses.
Finally (finally!), all the facts that were considered suitable answers are passed to a grammatical template to be sorted out and turned into flowing sentences. This process is pretty much the reverse of the fact extraction phase, except the syntax rules can be kept simpler. The template composes noun phrases, determines whether it can merge facts into summaries, where to use commas, pronouns, and link words. The end result is displayed as text, but internally everything is remembered in factual representation, because if the user decides to refer back to what Arckon said with “Why?”, things had better add up.
And my Axe!
There are more secondary support systems, like built-in common knowledge at ground level, common sense axioms* to handle ambiguity, a pronoun resolver that can handle several types of Winograd Schemas*, a primitive ethical subroutine, a bit of sarcasm detection*, gibberish detection, spelling correction, some math functions, a phonetic algorithm for rhyme, and so on. These were not high on the priority list however, so most only work half as good as they might with further development.
Arckon has always been a long-term project and work in progress. You can tell from the above descriptions that this is a highly complex system in a domain with plenty of stumbling blocks. The largest obstacle is still linguistic ambiguity. Arckon could learn a lot from reading Wikipedia articles for example, but would also misinterpret about 20% of it. As for Arckon’s overall intelligence, it’s about halfway the goal.
Throughout 2019 a limited version of Arckon was accessible online as a trial. It was clear that the system was not ready for prime time, especially with the general public’s high expectations in the areas of knowledge and self-awareness. The trial did not garner enough interest to warrant keeping it online, but some of the conversations it had were useful pointers for how to improve the program’s interaction in small ways. There are currently no plans to make the program publicly accessible again, but interested researchers and news outlets can contact me if they want to schedule a test of the program.
14 thoughts on “Introducing Arckon, conversational A.I.”
Of course, the first reply on this topic will skip all the appreciation for the huge amount of work that went into developing Arckon, writing this comprehensive and beautiful post and of course I will not mention how exceptional it is to build a conversational AI that is not a keyword-search-generic-response based system but a genuine attempt has been made to make it closer to general intelligence.
No, none of that – I will simply post a single sarcastic comment: The website says “Currently offline. Available on Mondays.” while I am accessing it on Monday……….
That’s quite alright. The problem with Mondays is that they start on a different time depending on where you are on the planet. I’ll put up a more accurate time indication eventually.
I have watching your entries on the forums and I am impressed. I like your approach to artificial intelligence in communication. I am looking forward to testing Arckon with the CIA test sets.
Thanks Toborman, I remember your mind map. I’m sorry if the results have been underwhelming: While you were testing I noticed in the logs that some of the program’s inference logic wasn’t hooked up to the online memory yet. Like applying a fact about all greeks to one particular greek. Should be fixed now.
Monday works fine for me. I’m retired, so I can test all day. I’ll start with tests 1A-1 through 10.
Alright, cool. I have a couple of tips for you: You can enter the premise and question in a single input as long as you use punctuation, e.g. “John is a Spartan. Spartans are greeks. Is John a greek?”, if you want to avoid contextual interference with the program’s responses inbetween.
If you contradict an earlier statement then the AI will gradually stop believing what else you tell it after that. The surest way to reset its memory is to end and start a new conversation.
Completed tests, Results are located on my website at http://vhmats.iwarp.com/rich_text_1.html
You put a lot of effort in this, I appreciate it. I see there are still a few flaws in my programming, but to be honest, Arckon will never just “forget” the way the test expects it to. To the program, contradicting statements indicate unreliable input, so they consolidate to decreased confidence instead of cancelling out completely. Are there other programs that you’ve tested?
Thank you for letting me use your intelligent program to evaluate my test questions. I have tested one other program, Harry Workman. I could use more programs to help my evaluation. The tests are still not validated and lack sophistication. Any suggestions you have would be appreciated. I value your opinion.
Well, the main thing that I noticed about your test is that its phrasing and expectations seem designed for 70’s expert systems with strict true-or-false logic and command templates. “Remember Tom is smart.” is shorthand and not grammatically correct like modern programs are designed for, it should be “Remember that Tom is smart”. “Remember” and “forget” are also a bit unusual to request: The first is superfluous and the second is not something you can literally command a human to do. In some other questions you have the commands “Verify”, “Prove” and “Locate”, which not all programs may interpret as expected because they’re not necessarily requesting a verbal response. You might instead ask direct questions, “Why…?” and “Where…?” to accommodate a broader range of programs.
Lastly, the test doesn’t account well for conversational AI, which many English-speaking AI will be. Arckon for instance deliberately avoids telling the user things that have already been said, because that wouldn’t be helpful in a real conversation. Sometimes you might just have to ask a straight question if you really want to know if the program remembers, when you don’t get the expected answer straight away.
I do like that your test is very methodical. I think it’s pretty sophisticated and its extensiveness prevents programs from consistently exploiting loopholes. Other candidates for your test that I can think of are perhaps Johnny http://www.vixia.fr/mylene/johnny.php , who is based on similar principles as Arckon, but seems to be a bit more particular about syntax, and Mitsuku.com, who I know can remember and draw inferences, but I don’t know how consistently those abilities are integrated.
Thank you for your comments. That’s exactly what I need`to improve this process. I will get to work making changes. I’m looking forward to establishing a standard for evaluating mental acuity.
I’d like your permission to add your suggestions to the results.
Sure, you may add my suggestions, with the exclusion of my descriptions of Johnny and Mitsuku as I think would be appropriate.
I have made changes to the test and evaluations. Check out the new results. Thanks again for helping improve the test. I expect more changes as I test more candidates.