• 4 Posts
  • 161 Comments
Joined 2 years ago
cake
Cake day: June 10th, 2024

help-circle
  • balsoft@lemmy.mltoTechnology@lemmy.world•How AI text watermarking works: a visual guide
    link
    fedilink
    English
    arrow-up
    4
    arrow-down
    1
    ·
    edit-2
    3 months ago

    Yeah, still, I wouldn’t claim “as no human would end up falling into that”, given that it may not be that unlikely to find at least one human who displays similar writing the more humans you involve.

    No, it is actually statistically impossible for a human to replicate this on sufficiently long runs of text.

    This is not about replicating writing like a model. This is basically about guessing which words to pick from the list of suitable words based on a rule that you don’t know (because the key is secret).

    To reduce this to the simplest possible example, imagine you are writing a “text” from just two letters: “a” and “b”. Let’s say for convenience that the text is supposed to be random. So the text would look something like “ababaaabbababbbabababaabbbabaababbaaabbabaabbaaaaabaaabbbaabaabababbabbbbbbbbabbabaabbbbbbbaabbabaab”

    (generated with '''.join(random.choice(['a', 'b']) for i in range(0, 50)))

    The watermarking works as follows: the model owner holds a key, and then uses that key to influence the random choices between “a” and “b” somehow, in a context-dependent way. The actual algorithm is quite complicated, but for simplicity let’s just say we have a secret pattern which biases the random choice towards it. In order to see the exaggerated results, let’s say the secret key is “aaaabbbb” (of course this is a bad secret key, once again just an example), and that the bias is strong (let’s say 80%). So this would mean that the first four letters in our text are more likely to be “a”, the next four letters are more likely to be “b”, then the next four letters are more likely to be “a”, and so on.

    Then the text would look something like “aaaabaabaabaabbbabaaaabbaaaababbbaaaabbbabaabbbbaaabaabbaaaaabbbabababbaaaaabbbbaaabbbbbaaaaababaaba”.

    (generated with ''.join(random.choice(['a', 'b'] + ([key[i % len(key)]] * 3)) for i in range(0, 100)))

    You can see visually that the secret key has affected the text. Of course in this example even if you didn’t know the secret key you could probably figure it out, in reality the algorithm is way more complicated than that, relying on cryptography, so you wouldn’t be able to know the secret key or see that the string has been biased at all.

    If the text is long enough, and you know the secret key, you can guarantee that the text was generated with it. In our examples, the letters in the text match our key 77% of the time. The probability of an actual random algorithm generating a text like that is already very low, despite the base entropy being only 100 bits. If my math is correct, for our example the p-value is 2.7 * 10⁻⁸, or about 0.00000027%. I would bet a hungy that the text was generated by our watermarking algorithm, with odds like these!

    Of course we did exaggerate the bias and our base algorithm was random. In reality the bias is smaller, the algorithm for determining the likelihoods of possible next tokens is very complicated (it’s the LLM itself), and the algorithm for determining which token to bias is also way more complicated (involving cryptography and real secret keys). That said, hopefully it should help you understand why, for sufficiently long texts, this fingerprinting is just not possible to be replicated by humans.


  • In the very old times, it didn’t matter too much whether you have jus soli or jus sanguinis. People rarely moved between villages, let alone between nation states, so the cases when they did could be handled individually. Still, jus soli probably made a bit more sense back then, because if someone took the great effort to come to your country they probably really wanted to live there and deserved citizenship.

    In the americas, the rationale used to be genocide. If your state is started & built on europeans eradicating the indigenous population, you’d want to only give citizenship to those europeans. The 14th amendment in the US was not about birthright citizenship per se, it was about extending the existing citizenship system to some non-white people - note that indigenous victims were not originally included!

    Nowadays the reason is humanitarian. If a person is born in your territory, it is cruel to make their life hell just because their parents are not citizens. And I do get that immigration is now easier than ever, but I still think some variation of a non-racist jus soli is a good idea.


  • balsoft@lemmy.mltomemes@lemmy.world•I was there Gandalf...
    link
    fedilink
    arrow-up
    2
    ·
    edit-2
    3 months ago

    I am almost certain my projector actually outputs 1080p. I can tell the difference between 1080p movies and 720p movies, I just don’t care too much.

    I think I might be weird because I also notice, but don’t care about the difference between 60Hz and 90/120/240Hz for my desktop monitors or my phone screen. I just leave everything at 60Hz which allegedly saves a bit of power. Other people tell me I’m insane for doing that and that they’d never come back to a 60Hz monitor.


  • balsoft@lemmy.mltomemes@lemmy.world•I was there Gandalf...
    link
    fedilink
    arrow-up
    5
    ·
    edit-2
    3 months ago

    I’m watching on a 2m wide projector screen, and honestly for most movies high-bitrate 720p is still acceptable. Heck, if the movie is good enough I’d even watch it in DVD quality (which is 480p). High-bitrate 1080p is nicer of course, but usually not enough to waste storage space on it. I’d much rather see 3-4 movies in 720p than 1 movie in 1080p (which is about the size difference you get).

    I guess different people have different preferences, but for me movies are not about examining the pimples on character’s faces. It’s about temporarily transporting me to a different reality, starting some in-depth discussion with whoever I’m watching it with, or just making me think about some area of life. Don’t need 1080p for any of that, my brain is more than good enough at suspending disbelief with a million pixels at any one time.

    Heck, some of my favorite movies have been shot on consumer-grade cameras and film. The medium doesn’t matter as much as the direction, acting, composition and cinematography in general.


  • balsoft@lemmy.mltomemes@lemmy.world•Small town cops really do suck
    link
    fedilink
    arrow-up
    6
    arrow-down
    1
    ·
    3 months ago

    Are there any actual dwellings around the road? If so, those speed limits are quite reasonable or even on the high end. In Europe it is 50 km/h (30 mph) if there are houses around a road, and 30 km/h (18 mph) for actual town streets. The streets are also specifically designed to be narrower and there are a lot of traffic calming measures to make drivers naturally want to drive the speed limit. This way it is safe & pleasant for townsfolk to cross the street, the town doesn’t end up split in two by the highway, and if you’re a sane driver you won’t get fined even if you accidentally miss a speed limit sign.

    Perhaps the best solution is to make the highways go around towns completely to avoid all of this. (and then we can also work towards banning cars in populated areas)


  • balsoft@lemmy.mltomemes@lemmy.world•Well, I tried.
    link
    fedilink
    arrow-up
    1
    ·
    3 months ago

    Actually you don’t really have to “flatten” anything, the parking lots are already flat enough. Remove asphalt and build mixed-use housing there. Take away a couple lanes from cars and build a tram line or at least a dedicated bus lane. It’s not that difficult if there’s political willpower for it.


  • balsoft@lemmy.mltomemes@lemmy.world•nur Bahnhof verstehen
    link
    fedilink
    arrow-up
    10
    ·
    edit-2
    3 months ago

    Don’t know about Farsi, but in Russian you would have to declare your knowledge and specificity of the subject. You can transmit some of that information by changing word emphasis, but it’s not very clear.

    As for your sentences I would translate them as:

    1. The girl downstairs was playing violin → Девушка [girl] внизу [downstairs] играла [was playing] на скрипке [violin].
    2. A girl downstairs was playing violin - Внизу [downstairs] девушка [girl] играла [was playing] на скрипке [violin].

    The first one puts more emphasis on the “girl”. The second one puts more emphasis on “downstairs”, almost suggesting that the identity of the girl doesn’t matter, kinda “some girl who happened to be downstairs”. But it is not as explicit as a definite/indefinite article for sure.

    If you want to make it certain that the identity of the girl doesn’t matter, you could use an adjective to that extent: Внизу [downstairs] какая-то девушка [some girl] играла [was playing] на скрипке [violin].

    If you want to make it certain that both you and the person you’re talking to know about the girl, you can use a determiner: Та девушка [that girl] внизу [downstairs] играла [was playing] на скрипке [violin].








  • balsoft@lemmy.mltoLinux@lemmy.ml•Is multi user nix safe?
    link
    fedilink
    arrow-up
    3
    ·
    edit-2
    3 months ago

    I’d like to note that the 4 points are not separate issues, they are all requirements in order for an attacker to get access.

    I’m not too worried about hash squatting, I guess there’s no way around it, but other ways of managing packages have way worse flaws than that.

    “hash squatting” is not really an issue on its own. Unless an attacker can breach the sandbox, given the same derivation hash, the resulting derivation output would be the same regardless of which user requested the build.

    I do want my users to be able to build packages. In most cases I imagine we’ll use pre built packages, but often packages available are not optimized for certain architectures.

    Actually, Nix without the ability to build derivations would just be useless. In Nix, everything is a derivation, even if you just want to have multiple packages installed into your user profile at once, behind the scenes it’s a derivation which simply combines binaries from multiple other derivations, which will not be available in the remote cache and needs to be “built” (in the most trivial sense possible) locally.

    How does the nix sandbox work? Just so I know what I’m working with. Do you know where I can read about it?

    You can think of the Nix sandbox as a kind of lightweight container, into which Nix mounts all the dependencies needed to build the derivation (including source code and such), and then runs the builder command. On Linux it is using User Namespaces (and a few other sandboxing tools), in a similar way to what Docker does (in fact there is some work to use runc as a sandbox backend but it’s not usable yet). I don’t know if it’s described in a lot of detail anywhere, but a brief description is available in the manual.

    Does root actively run installed packages at any point? If that is the case I’d be a bit wary to use nix for this purpose.

    Once again, just random packages in Nix Store are not “installed” in any real sense. In order for any user (incl. root) to install/run something, they have to be the one explicitly doing it, e.g. via nix profile install or nix shell. That also means that if you’re not using NixOS, you can actually just not do anything as a root user and then no code from /nix/store will be executed with root privileges at all.

    To reiterate, unless your threat model involves sophisticated attackers[1], users installing packages with Nix is as secure as them just building code from source manually, or dropping binaries into their own $HOME/.local/bin. One user installing a package into their profile does not affect other users or root in any way.

    From what I can tell from your other replies, you are setting up a common build server with your coworkers. In that case I’d say it’s 100% fine, both workplaces where I’ve been doing Nix work had a server like this, there were never any issues with privilege escalation or malware spreading between users.


    1. The attackers need to have access to an unpatched 0-day Nix sandbox breach or a critical nix-daemon issue, which is very unlikely. If that makes you feel any safer, I know that Nix has been scanned for vulnerabilities by at least one frontier-class LLM with no such vulnerabilities discovered, so just a script-kiddie with access to ChatGPT won’t be able to find anything like that. ↩︎


  • balsoft@lemmy.mltoLinux@lemmy.ml•Is multi user nix safe?
    link
    fedilink
    arrow-up
    9
    ·
    edit-2
    3 months ago

    Assuming you’re talking about Nix the package manager,

    Nothing is actually “installed as root”, except for stuff that the root user installs themselves. The Nix Store (where all the “derivations”/packages are stored) can be thought of more as a cache, just because a package is there doesn’t mean it’s used anywhere. Users don’t get to choose the cache “key” (i.e. the directory name in /nix/store) either - it is determined by all the build instructions and dependencies needed to build the package, and Nix doesn’t (well, at the very least shouldn’t) give users any control about the package build process after it starts.

    When users install something, Nix fetches or builds that package into /nix/store - which doesn’t affect other users in any way - and then just symlinks that particular package into some user-owned directory in their $PATH (e.g. $HOME/.local/share/nix/profiles/default or so), which also doesn’t affect root or other users in any way.

    So, basically, if some user installs malware only they are affected - the fact that it’s also in the shared /nix/store is irrelevant since there’s nothing in other user’s profiles or $PATH or whatever that references it in any way.

    The most likely vulnerability is something like this:

    1. The attacker guesses the nix store path of some package that root will use in the future (e.g. glibc from a more up-to-date Nixpkgs version compared to what root currently uses) - note that users can’t replace a derivation that’s already in the store, so they need to guess a future derivation path
    2. The attacker forces Nix to build that package from source instead of fetching from a trusted substituter
    3. The attacker finds a way to breach Nix’s sandbox during the build and inject their own backdoor into the resulting package ← this is the difficult part, there are currently no such known sandbox holes
    4. The attacker then waits until root starts using that package in the store, at which point the backdoor becomes actively ran as root.

    I must add that this is theoretical and I don’t think has ever happened in practice on a multi-user system.

    Sorry, I’m pretty bad at explaining stuff, hopefully it makes some sense :)




  • But also the problem is that they cheat.

    Using AI is easy, as you note. You don’t need to go to college to learn to do that.

    College can give you many useful skills, mostly related to gathering and processing information, and applying it to solve various issues. If you just copy-paste every question into AI and then copy-paste the answer back, you will learn absolutely nothing.

    LLMs hallucinate and there’s nothing we can do to stop them. No amount of QA will fix that fundamental issue of “the LLM is somewhat likely to confidently produce incorrect information”, if you need the answer to be correct you have to do the research yourself, maybe based on the sources the LLM provided, not just sanity-check the output. This is a very useful skill to have for many jobs involving thinking or finding stuff out.

    And even if we fix the hallucinations at some point by switching to a different tech (LOL), I do think it’s useful for people to keep some mental skills. Otherwise we will end up with a stupidity and/or dementia epidemics, similar to cars causing an obesity epidemic.


  • Ok, so there’s no statistically rigorous review yet, but 73% of LLM-generated apps submitted to FlatHub were abandoned within 6 months. This is about the abandonment rate you should expect to see on any newly announced LLM-written project, and this figure appears to be much higher than hand-written projects (e.g. this study, while not directly comparable, shows a 16% abandonment rate).

    The reason for this is sort of obvious to me: an LLM makes it really easy to write code that makes an app work, therefore the threshold of involvement/interest needed to “make an app” is much lower. This in turn removes the selection force that makes software authors likely to be interested in, and thus put in the effort (or tokens) to maintain their software going forward. (this is putting aside the fact that LLMs are still horrible at software architecture and quite bad at infrastructure, making the long-term maintenance more difficult).

    I don’t doubt that some people will have a lot of passion for their vibecoded thing, and will keep it going for a while. This might even be the case here given the author went out of their way to post about it on Lemmy and respond to comments, seemingly without an LLM in the loop at that stage of the process. But it’s also important to disclose your LLM use so others can adjust their statistical expectations.

    anecdotally, I’ve built a lot of projects and abandoned them because I ran out of time. With an LLM to reduce the input effort I actually think I would’ve maintained them longer.

    Likely you would have also built more projects if you had access to an LLM, thus giving you less time to spend maintaining each individual one. Which is kind of the problem here…