You can set up rules to allow SSH connections for specific IP addresses.
Log in to the server and enable the firewall with the command `ufw enable` and type `y`.
Add a rule to allow SSH connections from a specific IP address. Enter `ufw allow from <ip address> to any port 22`.
Deny all other SSH connections. Enter the command `ufw deny 22`.
Check the rules with the command `ufw status`.
Now your server can only be accessed via SSH from the designated IP address.