{"id":55,"date":"2024-02-28T00:07:39","date_gmt":"2024-02-28T00:07:39","guid":{"rendered":"https:\/\/caltechaia.org\/?page_id=55"},"modified":"2024-02-28T00:17:57","modified_gmt":"2024-02-28T00:17:57","slug":"new-to-ai-safety-start-here","status":"publish","type":"page","link":"https:\/\/caltechaia.org\/index.php\/new-to-ai-safety-start-here\/","title":{"rendered":"Why care about AI Safety?"},"content":{"rendered":"\n

In recent years, we\u2019ve seen AI exceed our expectations in a wide variety of domains \u2014 including playing Go, composing human-like text, writing code, and modeling protein folding. It may not be long until we create AI systems that are much more capable than humans<\/a> at solving most cognitive problems.<\/p>\n\n\n\n

Such powerful systems could bring great benefits, but if their goals don\u2019t line up with human values, they could also cause unprecedented disasters<\/a>, and even human extinction<\/a>.<\/a><\/p>\n\n\n\n

Rapid progress in the capabilities of current AI systems has pushed the topic of existential risk<\/a>  from AI into the mainstream<\/a>. The abilities that GPT-4<\/a> and other recent systems display used to seem out of reach in the foreseeable future. The leading AI labs today are aiming to create \u201cartificial general intelligence<\/a>\u201d in the not-too-distant future, and many top researchers<\/a> are warning about its dangers<\/a>.<\/p>\n\n\n\n

Even when AI becomes as smart as humans in most domains, there\u2019s no known impediment to it continuing to get smarter: just as current AI vastly outperforms us at arithmetic, future AI will vastly outperform us<\/a> in science, technology, economic competition, and strategy. When AI becomes capable of replacing humans for most of the work involved in AI research, this will accelerate such research, potentially resulting in a \u201csuperintelligence\u201d in a short time<\/a>.<\/a><\/p>\n\n\n\n

A superintelligent AI could be incredibly useful in the quest for human flourishing<\/a>, if its actions are in line with human values. But it\u2019s not guaranteed<\/a> that they will be. A central concern of AI safety is making sure that AI systems try to do what we want, and that they keep doing so even if their circumstances change fundamentally<\/a> \u2013 for example, if their cognitive capabilities exceed those of humans. This is called the \u201cAI alignment<\/a> problem\u201d, and it\u2019s widely regarded as unsolved and difficult<\/a>.<\/p>\n\n\n\n

AI alignment researchers haven\u2019t figured out how to take an objective and ensure that a powerful AI will reliably pursue that exact objective. The way the most capable systems are trained today makes it hard to understand how they even work<\/a>. The research community has been working on these problems, trying to invent techniques and concepts for building safe systems<\/a>.<\/p>\n\n\n\n

It\u2019s unclear whether these problems can be solved before a misaligned system causes an irreversible catastrophe<\/a>. However, success becomes more likely if more people make well-informed efforts to help<\/a>. We made this site to help people understand the challenges at hand and the solutions being worked on. The related questions below are a good place to start learning more, or you can enter your questions into the search bar.<\/p>\n\n\n\n

\n
\"\"<\/figure>\n<\/figure>\n","protected":false},"excerpt":{"rendered":"

In recent years, we\u2019ve seen AI exceed our expectations in a wide variety of domains \u2014 including playing Go, composing human-like text, writing code, and modeling protein folding. It may not be long until we create AI systems that are much more capable than humans at solving most cognitive problems. Such powerful systems could bring great benefits, […]<\/p>\n","protected":false},"author":1,"featured_media":0,"parent":0,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"om_disable_all_campaigns":false,"footnotes":""},"class_list":["post-55","page","type-page","status-publish","hentry"],"aioseo_notices":[],"_links":{"self":[{"href":"https:\/\/caltechaia.org\/index.php\/wp-json\/wp\/v2\/pages\/55","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/caltechaia.org\/index.php\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/caltechaia.org\/index.php\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/caltechaia.org\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/caltechaia.org\/index.php\/wp-json\/wp\/v2\/comments?post=55"}],"version-history":[{"count":2,"href":"https:\/\/caltechaia.org\/index.php\/wp-json\/wp\/v2\/pages\/55\/revisions"}],"predecessor-version":[{"id":74,"href":"https:\/\/caltechaia.org\/index.php\/wp-json\/wp\/v2\/pages\/55\/revisions\/74"}],"wp:attachment":[{"href":"https:\/\/caltechaia.org\/index.php\/wp-json\/wp\/v2\/media?parent=55"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}