×
all 102 comments

[–]LoserOtakuNerd 106 points107 points  (11 children)

I've followed your work for years now and I am always impressed by your work ethic and skill. Every time you publish any work I love looking at it and studying it. Heck, I read the Atmosphere release changelog like a novel when I see there's a new one. Thanks for your contributions to the world of experimenting, preservation, and the technical community at large.

[–]SciresM[S] 65 points66 points  (9 children)

Thanks!

Atmosphere is very much a personal passion project/a labor of love, and it means a lot to me to hear stuff like this.

[–]djh816 10 points11 points  (8 children)

Seconded on appreciation for Atmposphere! I recently realized I played very few switch exclusive games and have since mainly been using my switch with Atmosphere and the Moonlight streaming port. Great work and from the 30 mins or so I watched of the kernel dev on twitch (which was way over my head), I really appreciate you sharing your work and explaining the process!

[–]g0atbutt 1 point2 points  (7 children)

I didn’t realize there was a moonlight port! Could you possibly link that? Thanks /u/djh816

[–]djh816 5 points6 points  (6 children)

https://github.com/rock88/moonlight-nx/releases

By far the lowest latency streaming I've gotten on a portable device. I played through some Halo campaign missions over 5ghz wifi recently and had no issues, definitely playable. I'd leave the settings for bitrate on default (10-12mbps if I recall correctly) but maybe someone with better signal could get some higher quality with tweaking that. Mine started stuttering if I tried to raise bitrate and honestly it doesn't need more than the default.

Edit: One other note.. I tried SkyNX before this but the added requirement of another PC app and the slow latency made me look for other options. Moonlight-NX uses my existing NVIDIA gamestream and is far better on latency (props to SkyNX though for a cool app!)

[–]g0atbutt 2 points3 points  (0 children)

Awesome, thanks for the info!

[–][deleted] 1 point2 points  (4 children)

how do you manage to enable network/wifi on your hacked switch without risks of getting banned? I have disabled wifi to prevent getting banned.

[–]djh816 1 point2 points  (3 children)

I use 90dns and never had an issue. Also I never linked a Nintendo account on emunand side

[–][deleted] 1 point2 points  (1 child)

ok thx! will try that. I was always afraid allowing network access to my emunand

[–]djh816 1 point2 points  (0 children)

I made sure to wipe and then apply 90dns to switch before connection to network by specifying during manual network setup. Good luck

[–]Schadrach 0 points1 point  (0 children)

You can also use incognito to change your prodinfo to have a fake serial number such as all zeros in the emunand. That way the fake serial number is the ban target.

[–][deleted] -1 points0 points  (0 children)

it’s too bad he’s a homophobe

[–]Kruzenstern 22 points23 points  (13 children)

I lack any programming knowledge myself but want to ask; how accurate would the emulation be if emu devs used your kernel as reference instead of reverse engineering the prorietary kernel? Just as accurate?

[–]SciresM[S] 28 points29 points  (12 children)

Just as accurate.

If you're familiar with the various "decompilation projects" for games that have popped up in the last year or two, this is kind of the same idea but for the kernel (and I'm not aiming to produce byte-for-byte identical binaries).

All the code reflects my honest best understanding (and implementation) of what Nintendo's kernel does.

[–]JediThug 7 points8 points  (10 children)

Is it really decompilation or just clean room reverse engineering? (I hope for legality reasons it's the latter, lol)

[–]SciresM[S] 25 points26 points  (9 children)

I am a reverse engineer and hacker. Frankly I don't think traditional clean room is viable -- you cannot precisely match the behavior of components (especially the scheduler and page table managers) without directly observing them.

[–]JediThug 6 points7 points  (4 children)

Well that's kinda what clean room is though, right? You're trying to get as close to its functionality while not essentially "copy-pasting" code from the target source. Then again, I'm not really that versed in RE (while the concept is really interesting to me).

[–]bringsyoufish 11 points12 points  (3 children)

The only good legal definition of clean room we have is from when IBM sued people for re-implementing the PC BIOS. It requires:

  1. Having team 1 RE the binary, write documentation about how it work without including any code, not even manually decompiled, from the original.

  2. Having a team 2 that do not talk to team 1 reimplement the functionality only based on team 1s documentation.

[–]DaveMurphy 5 points6 points  (2 children)

Do you have a source for this? I didn't know the implementation team couldn't talk to the RE team.

I think these rules are bit arbitrary tbh.

[–]bringsyoufish 8 points9 points  (1 child)

There are no rules for clean room, just legal precedence. That procedure was what Phoenix did and got away with. Other methods might work, but you'd have to get it past a court. From memory and IANAL. Probably a good jumping off point:

https://en.wikipedia.org/wiki/Clean_room_design

[–]enleeten 1 point2 points  (0 children)

Yeah unless you have a lot of cash to spend on the legal costs, it doesn't even matter if you could win in court.

Nintendo could keep you in court for years, even if they know they'd lose.

[–]DaveMurphy 5 points6 points  (0 children)

I think you could do it if you have good communication with the RE team. Definitely not going to be as efficient as observing and implementing yourself but not impossible.

[–]ThomasThaWankEngine -4 points-3 points  (2 children)

Ok so does it have some of the original code in it?

[–]masagrator 5 points6 points  (1 child)

No, because there is no available source code for kernel anywhere we are aware of to even take original code lines.

And from what I see in code SciresM is trying to make it so there is no chance of it being 1:1 with original source code if it leaks anytime soon while still doing it as optimized as possible.

[–]ThomasThaWankEngine 0 points1 point  (0 children)

Ok cool

[–]djh816 1 point2 points  (0 children)

From watching your twitch stream (and maybe again this is over my head lol), how do you choose which methods or functions to "stub"? Do you test kernels and see what the functions output or go based on the function names etc? Are the function names visible either from calls or decompilation of the kernel itself (do you figure these out? symbol tables?)? So many questions :)

[–]Caos2 11 points12 points  (0 children)

This is amazing work, thanks for sharing.

[–]rayman3003 10 points11 points  (0 children)

Thank u for ur hard work. (ღ˘⌣˘ღ)

[–]How2Smash 9 points10 points  (17 children)

As someone with interest in the technical side of this, can you explain what this is? This is a micro kernel distinct from the main OS's FreeBSD based kernel, correct? What technical details about Horizon do you find so fascinating?

Is there some documentation you have?

[–]SciresM[S] 23 points24 points  (16 children)

"Main OS's FreeBSD based kernel"

The Switch doesn't use FreeBSD at all, that was a rumor started by people who looked at copyright notices. This is the main kernel. The Switch runs a completely custom OS (Horizon) with a design totally different to FreeBSD/Linux.

(The copyright notice comes from Nintendo using sys/tree.h from FreeBSD for intrusive lists, but that's it.)

There's a list of syscalls on the wiki: https://switchbrew.org/wiki/SVC

Admittedly we're a little short on documentation. I was thinking about making a kernel wiki like google has for Fuscia/Zircon once I'm done implementing it :)

"What do you find so fascinating"

It is a completely unique microkernel with a cooperative (non-preemptive) scheduler. The kernel is secure -- so far as I can tell (as a reverse engineer and hacker), it has zero security bugs. They throw out years of backwards compatibility (they're not POSIX/UNIX), and they really, really benefit from it from a security and modularity PoV.

Horizon's the only meaningful RTOS with a microkernel that I'm aware of (other than Fuschia). Everything's in userland -- filesystems, gpu (and other device drivers). The OS is capability-based and conceptually all about lots of different processes/drivers ("system modules") that host microservices.

The fact that Nintendo designed such a rock-solid, modular, custom operating system for their consoles fascinates me.

I hope that answers your question :)

[–]How2Smash 10 points11 points  (1 child)

Huh. Do you think they made any tradeoffs with performance for security? Typically FUSE is slower than a kernel driver for a filesystem in my Linux experience.

Also is there a major way that they deviate from UNIX?

[–]SciresM[S] 11 points12 points  (0 children)

There are always trade-offs in that sense, to be honest.

The advantage to using a microkernel is security -- there's sufficiently little code that you can actually be confident it's all secure.

The disadvantage is that because things that would live in the kernel under other designs are now in userland, they have to do IPC to communicate -- and IPC has some overhead.

IPC is the hottest hot-path in a microkernel, correspondingly Nintendo marked every function involved in IPC as __attribute__((always_inline)), this was kind of a huge pain to reverse engineer as a result.

In addition, Nintendo implemented "SvcReplyAndReceive" as a single system call that allows a microservice server process to reply to and receive a new message in one invocation.

That said, there's actually less overhead than you think. Past of why FUSE is slower than a kernel driver for FS is because FUSE has to talk to the kernel to do filesystem stuff, so when you read a file you have your process -> FUSE -> kernel -> hardware. In comparison, on Horizon the kernel is completely uninvolved in filesystem management (it doesn't even have the sdmmc hardware mapped). Thus processes will do process -> FS system module process -> hardware.

Is there a major way that they deviate from UNIX

In UNIX, everything is a file. Communication happens over pipes.

In Horizon, everything is very distinctly not a file. There's no global filesystem paths the way that unix/linux have special /dev/whatever.

Pipes don't exist in Horizon -- all IPC is done via the horizon ipc ("HIPC") protocol.

UNIX/POSIX have stuff like fork() and child processes...but creating a process is an incredibly privileged operation in a capability-based operating system. Fork() is impossible to implement in Horizon, all threads are created via SvcCreateThread() instead. Child processes aren't a thing that exist.

[–]sunjay140 3 points4 points  (7 children)

Does this mean I'll never be able to run RetroArch on my Switch Lite since that requires a kernel exploit?

[–]masagrator 8 points9 points  (3 children)

But it works... Retroarch doesn't require kernel exploit. There is no kernel exploit in Horizon that we are aware of.

[–]sunjay140 2 points3 points  (2 children)

I'm not aware of any way to run unsigned code on Switch without some sort of exploit.

[–]masagrator 8 points9 points  (0 children)

Well, atmosphere is using exploit in RCM to run it's own bootloader and acquire all privileges. It's not kernel exploit.

Atmosphere can be run also with custom bootloader - look Hekate.

[–]PrimaCora 1 point2 points  (0 children)

Switch lite a hardware exploit like any other switch, well, more difficult than any other switch

[–]CompSciOrBustDev 4 points5 points  (2 children)

I'm no where near as knowledgeable as Scires but I'm involved in the Switch homebrew development scene. You can actually do that right now. The OS is completely secure but the hardware is not. Currently the only commercial mod chips are made by Team-Xecuter who some are opposed to for moral reasons but I'm using one of their chips right now and it works well. It works the same way as the RGH hack for the Xbox 360. Sooner or later I'm sure a clone will come along if you don't want to support TX.

Edit: Also it doesn't necessarily need a kernel exploit. See the rohan exploit for Switch firmware 3.0.0. I doubt we'll see another userland vulnerability as powerful as that again but at the time it allowed for running homebrew.

[–]leo60228 1 point2 points  (1 child)

For what it's worth, TX makes the only commercial modchip for current Switches. If your Switch is vulnerable to the Fusee Gelee exploit, there's plenty of modchips, many sold under the name RCMX86.

[–]CompSciOrBustDev 1 point2 points  (0 children)

This is correct but the guy above was asking about Switch Lites which all have bootroms that are invulnerable to f-g.

[–][deleted] 1 point2 points  (1 child)

Horizon's the only meaningful RTOS with a microkernel

Wasn't QNX like that?

[–]SciresM[S] 1 point2 points  (0 children)

Neat. TIL about QNX :)

[–]Rhed0x 9 points10 points  (0 children)

Awesome work.

This is a weird question that you probably get all the time but do you think it would be possible to get things to run on the Nvidia Shield Android TV? As far as I know it uses the exact same SOC.

[–]lucicam 9 points10 points  (0 children)

I'm actually really curious how do you even start with such a project? Did you dump the kernel from the switch and then by disassembling it you try to recreate the source code? Or how exactly do you start with such a project?

[–]chrisfu 17 points18 points  (5 children)

If Nintendo don't make an offer for you to go work for them after all the excellent work you've done, I'd be dumbfounded. It's an absolutely staggering software engineering feat, particularly at this point in the consoles life cycle.

Yes, Nintendo left the door open. SciresM entered and now knows where every nut, bolt and rivet is. They probably known Horizon better than any individual Nintendo dev by this point.

[–]smith7018 10 points11 points  (4 children)

Nintendo didn’t leave the door open; NVIDIA did. SciresM has stated that Horizon is actually pretty solid security-wise

[–]chrisfu 2 points3 points  (3 children)

Well, yep, but we're getting into semantics there. You could argue that Nintendo retaining the ability to easily trigger the bootloader recovery (RCM) was as bad as as burnt-in bootloader software exploit.

[–][deleted] 2 points3 points  (2 children)

But RCM is how they repair Switches.

[–]chrisfu 2 points3 points  (1 child)

Indeed it is. They made it too easy to trigger again; see NTRBootHax for the 3DS.

[–]ZachyCatGames 1 point2 points  (0 children)

RCM was intentionally made easily accessible so software issues could be fixed and/or the system can be tested via RCM without doing anything overly complicated (like taking the device apart).

What wasn't intentional is the massive bug that Nvidia left in RCM.

RCM should've been secure, but Nvidia fucked it up.

Thus, this is completely Nvidia's fault.

NTR recovery on 3ds was also intended to be secure, but was broken by sighax.

[–]JQuilty 9 points10 points  (3 children)

So is Horizon something new or based on Linux or BSD?

[–]SciresM[S] 17 points18 points  (2 children)

It is a completely novel design -- Horizon was originally designed for the 3DS, but a hard rewrite to solve all the design problems was done for the Switch.

It's a microkernel with a (mostly)-cooperative scheduler. It's totally original and I'm really impressed with it from a security point-of-view.

[–]MrGaytes 7 points8 points  (1 child)

This account has been scrubbed in response to Reddit's API changes. I will NOT use their crap app. I've had this account since 2014 and 10k Karma. I never cared about reddit. Reddit thinks it has more power than it actually does.

If you want to change to a decentralized platform like Lemmy, you can find helpful information about it here: https://join-lemmy.org/ https://github.com/maltfield/awesome-lemmy-instances

Good riddance.

[–]SciresM[S] 10 points11 points  (0 children)

See my comment here for more details.

[–]ScarletSpeedster 8 points9 points  (0 children)

You’ve come a long way from writing Pokémon translation projects a decade ago, the work here is nothing short of amazing. As someone who used to be a part of the RE scene back then, it’s come so far. You’ve turned these projects from something that was often drama filled into some of the most professional public work! Thanks for all you do.

[–]stuken 5 points6 points  (4 children)

The dual address rw/rx jitbuffer we have with libnx has been a bit of a pain in the past. Any plans to allow for rwx memory mapping as part of mesosphere?

[–]SciresM[S] 11 points12 points  (3 children)

I am pretty opposed to rwx mappings - besides the technical issues (the kernel has a pretty strict memory state model and is full of assertions/panics when transitioning memory states that I would like to accurately reflect), I think that they're kind of unnecessary design wise. In order to safely do JIT you necessarily need to make sure no threads are executing pages that are being modified, and make sure that the cache is invalidated/managed correctly before executing the modified code. This means you already require some kind of "transition/prepare to write", "transition/prepare to execute" API, and W^X is just a mechanism enforces user code correctness.

That said, I can see how having the RW- view and R-X view be at different addresses can be annoying. I could definitely be amenable to making sure it's possible to have the addresses be the same.

[–]DCNick3 0 points1 point  (2 children)

Isn't it already possible with `MapPhysicalMemory` + `SetProcessMemoryPermission`? Or some memory state magic stops it from working?

[–]SciresM[S] 0 points1 point  (1 child)

Memory permission SVCs do not allow simultaneous W and X. You can have two mappings of the same physical pages (one W, one X), or can transition the same mapping between RW- and R-X, but you can't have one mapping that has both.

Fwiw even besides getting the kernel to set the bits in the page table, the secure monitor configures hardware enforcement of WX, so page table entries with both are invalid.

[–]DCNick3 0 points1 point  (0 children)

Oh, I see. Thanks for the explanation

[–][deleted] 5 points6 points  (0 children)

This is incredible. Nice work.

[–]AstronomerOfNyx 3 points4 points  (0 children)

Thank you for sharing. I didn't realize Nintendo did so much of this in house. I, very ignorantly, assumed this was something they'd avoid doing themselves because their OS in the past have been so lackluster. There's also the accusation that they stole the rail tech, which just leant to the idea for me that they were technologically incapable. Kind of ironic because from a user standpoint I've always loved Sony's OS but they did actually use Freebase without acknowledging it, whereas Nintendo, apparently, has been hard at work securing the software on their own terms. Strange that they would put such care into the software side with a known hardware vulnerability. Any thoughts as to why they may have ignored that? Maybe they were too far into development when the hardware exploit was made public.

[–][deleted] 2 points3 points  (0 children)

Excellent work 👍

[–]pr0_c0d3 2 points3 points  (0 children)

I love open source projects like this

[–][deleted] 2 points3 points  (2 children)

So is the plan for this to be a drop in replacement for horizon, just for the switch, or are you looking to make it portable to other arm64 devices?

[–]SciresM[S] 7 points8 points  (1 child)

For now I've just been targeting the switch.

I'd like to get the kernel itself running on both some other arm/arm64 devices I own and on my x64 PC so that I can play around with it, but the kernel is only one part of a pretty large OS :)

[–][deleted] 0 points1 point  (0 children)

Yeah, the userland is huge.

[–]SaccharomycesSapiens 5 points6 points  (2 children)

Sorry if I'm completely misunderstanding what this is, but does this mean you could run Switch games on a normal Nvidia Shield with 4 GB of RAM by booting it into this OS?

[–]Sol33t303 10 points11 points  (0 children)

Unfortunately no, theres a lot more then just a kernel in a system, and although the kernel is a very big and very important part, the rest still has to be reverse engineered and ported to MAYBE port the kernel to other hardware and run games.

It is not a full OS, just a kernel.

[–]Teethpasta -1 points0 points  (0 children)

The nvidia shield comes with a max of 3gb of ram.

[–]OrShUnderscore 1 point2 points  (0 children)

I'm techy so I know what most of these words mean, but not everything. However this is awesome! Good work :D

[–]nemrod9 1 point2 points  (0 children)

You are a god. Thanks a lot for your work

[–]noxiousninja 1 point2 points  (1 child)

Impressive work!

How much has the kernel changed over the years/how much maintenance do you think will be required going forward to stay compatible with future OS releases?

Also, how optimized/obfuscated is the code? Is it pretty easy to pick out bits that changed between versions?

[–]SciresM[S] 11 points12 points  (0 children)

Thanks!

The kernel has changed moderately over the years, but they've kind of settled down design-wise and I think it's unlikely that there'll be anything too major.

The kernel only changes in "major" system updates (Nintendo uses semver), here's the changelog I wrote for the most recent system update in April.

Maintenance is a minor concern, relatively speaking. There are ~3-4 major updates per year. It takes me about 2-3 days to "fully" difference the kernel (produce a reverse engineering database with every function labeled), and a few hours to identify all the "important" differences. I have a bunch of scripts for automatically labeling stuff (by parsing the SVC tables and .got) that make stuff easier.

It took me about two days to implement all the changes from 10.0.0 into my codebase.

My expectation is that a few times a year, the kernel will update, I'll have to spend about a week differencing + updating my code to be accurate, and probably about half that time to have new firmwares booting but not perfectly accurate.


The code is build with clang at -O2. They don't use any obfuscators.

Newer kernels are harder to reverse engineer than old ones because they added __attribute__((always_inline)) to all the spinlock/synchronization primitives, and so you kind of need to learn to recognize those patterns and "see through them" to just go "oh yeah that's the constructor for KScopedLightLock".

Older kernels didn't have __attribute__((always_inline)) and so these functions were never inlined, which helped a lot for getting my bearings when I was preparing for the project.

[–]akc8012 1 point2 points  (0 children)

nice stuff 👍

[–]Edward_Elric64 1 point2 points  (0 children)

Wow. Absolutely incredible!

[–]Snowflake Devron975 1 point2 points  (0 children)

Any plans to backport your work on mesosphere to the 3DS?

[–]False_Cartoonist 1 point2 points  (0 children)

This is the coolest thing I've seen in a while, and I can't express my gratitude enough that we have someone so forward-thinking as the main dev for Atmosphere. The implications of this are absolutely huge, even just in terms of the reverse engineering time this is going to save down the road. Bravo!

[–]rbooris 0 points1 point  (2 children)

Do you know mesosphere is also the name of a project under Apache Mesos? It is arguably dead but just sharing to check if there is any connection: https://d2iq.com/solutions/mesosphere and https://en.wikipedia.org/wiki/Apache_Mesos?wprov=sfti1

Thank you for all your work it is fascinating to see your progress and development

[–]SciresM[S] 0 points1 point  (1 child)

Heh, I don't think I was aware of that.

Either way, I don't think that impacts my naming choice -- it's no reason not to name Atmosphere's components after different regions of the atmosphere, heh.

[–]rbooris 0 points1 point  (0 children)

Agreed - merely an FYI that’s it

[–]Defender of the Seascrabycowman123 0 points1 point  (1 child)

Does this mean we can or will be able to run Atmosphère without relying on Horizon? Could this open up the possibility of running games that require newer firmware (and a newer kernel) even without using Nintendo’s newer firmware? Is there an option now to force Atmosphère to run without any Horizon code?

[–]VirtualAdepts 2 points3 points  (0 children)

You can already patch games to use a lower FW.

From my understanding. Atmosphere is a CFW, but not a full firmware replacement. Mesosphere is a kernal replacement. Still need a Horizon replacement.

But everything I said, could be wrong. Except the part that you can patch switch games to use a lower FW already

[–]ChrisX930 0 points1 point  (0 children)

You have my Upvote, sir

[–]MAli800 0 points1 point  (0 children)

You're a great dev SciresM, keep up the good work and stay safe. Great work!

[–]4-Fluoroamphetamine 0 points1 point  (0 children)

Incredible work.

[–]KingOfCannabis420 0 points1 point  (0 children)

Great work. You’re clearly dedicated. Forgive me for I’m not program savvy. Theoretically, what would the implications of this be? Purely for emulation? Or are you essentially trying to duplicate Horizon in its entirety & improve it? What is the end goal?

[–][deleted] 0 points1 point  (0 children)

will use for piracy

[–]ETGXX1 0 points1 point  (0 children)

Does it wir on mariko models without the chipset?

[–]DustyLance 0 points1 point  (0 children)

You are insane !

[–]alycrafticus 0 points1 point  (4 children)

Firstly, thank you for your work over the past few years, it's massively appreciated.
Secondly, I have some practical questions by rebuilding the kernel from scratch it means you can omit and add features, does this mean down the line we would have the ability to code ways of playing multiplayer without Nintendo servers, and will this make it harder for nintendo to discover hacked consoles as there will be no implementation of nintendo code talking to their servers etc?

[–]Sandstar101Rom 0 points1 point  (3 children)

Nintendo uses aauth IIRC to prevent this.

[–]alycrafticus 0 points1 point  (2 children)

But given it won't be running Nintendo back end would this not be emulatable? Fake authenticate and redirect server address? Won't be much good for games hosted on Nintendo servers but p2p based games would work I imagine?

[–]Sandstar101Rom 0 points1 point  (1 child)

But given it won't be running Nintendo back end would this not be emulatable? Fake authenticate and redirect server address? Won't be much good for games hosted on Nintendo servers but p2p based games would work I imagine?

Thats not what mesosphere is. Mesosphere is a kernel reimplementation. It has nothing to do with Nintendo's authentication code and this does not make it easier. All games and servers use this thing called RSA to prevent fake redirects and stuff.

[–]alycrafticus 0 points1 point  (0 children)

Ah OK, thanks for clearing that up for me :)

[–]BennyAlex98 0 points1 point  (0 children)

If your using this kernel on your switch, does it mean you don't have to hardmod it to coldboot into cfw?

[–]TransGirlInCharge 0 points1 point  (0 children)

Thank you. :)

[–]HonorMyBeetus -1 points0 points  (0 children)

Will we be able to run mesosphere instead of horizon on a switch?