Let’s start by examining what FTP is and when it was introduced. The File Transfer Protocol (FTP) allows an organization to host file storage and enables users to log in with usernames and passwords to download and upload files to the server. FTP is widely used in the identity and data provider industry to transfer large files between identity providers, brands, and publishers. While FTP is a solution relied upon in day-to-day business, FTP is not a new solution–it’s been around quite a while!<\/span><\/p>\n The problem of transferring files between computers predates the modern internet and the World Wide Web. Shortly after the first early computers were connected through ARPANET, the first FTP protocol was created in 1971 (<\/span>RFC 114<\/span><\/a>) to enable the transfer of files from one computer to another. Over the past 50 years, this protocol has evolved, transformed, and changed into today’s two widely used protocols. They are most commonly known as the File Transfer Protocol (<\/span>FTP RFC 989<\/span><\/a>) and the SSH File Transfer Protocol (<\/span>SFTP Internet Draft Standard<\/span><\/a>).\u00a0<\/span><\/p>\n While SFTP addresses many more security concerns than the original FTP, we knew we could offer our customers even more protection by default. By building on top of existing standards and open source libraries (SFTP and the <\/span>golang sftp library<\/span><\/a>), FullContact introduced a new server called CipherFTP (CFTP). CFTP provides all of the protection of SFTP while introducing new secure-by-design features that usually need to be manually implemented or bolted onto SFTP. To examine the differences between FTP, SFTP, and CFTP, we will explore the security challenges that need to be addressed when transferring sensitive data over insecure networks:<\/span><\/p>\n Authentication is how a user proves they are who they claim to be. Once a server has established a user’s verified identity, it can grant them appropriate access to systems and data.\u00a0<\/span><\/p>\n The most common way to establish identity is through a username and password. While passwords have several shortcomings, one of the most underlying issues is that they rely on shared knowledge.\u00a0<\/span><\/p>\n To illustrate the problem of shared knowledge and FTP passwords specifically consider the following examples:<\/span><\/p>\n Another common way of verifying a user’s identity is through public\/private key cryptography. Suppose a user is assigned a public\/private keypair. In that case, any user or system can encrypt data using the public key, which can only be decrypted using the corresponding private key. While public keys can be distributed (hence the name “public”), private keys are considered secret and strictly guarded by the user.\u00a0<\/span><\/p>\n The original FTP server supported only simple usernames and passwords, whereas SFTP supports both usernames and passwords and public\/private key authentication.\u00a0\u00a0<\/span><\/p>\n CFTP also supports both username\/password and public\/private key authentication. While FTP and SFTP require user accounts to be created and managed by an administrator (using LDAP or other means), CFTP allows users to create and manage their own accounts. CFTP additionally provides a secure way for users from a shared organization to access shared data while using distinct identities.<\/span><\/p>\n Whenever you are connecting to a server on the public internet, whether that is a web server or a FTP server, there is the chance you might not be connecting to who you think you are connecting to. Imagine the scenario below where Sally thinks she is connecting to <\/span>ftp.data.inc<\/span>. In reality, Evil Bill has set up a new server that intercepts the original commands, logs them, and forwards them to the real <\/span>ftp.data.inc<\/span>. At this point, Evil Bill has all of the data and any user data and passwords sent.<\/span><\/p>\n While this was (and still is) a real problem with the original FTP, SFTP solves this issue the same way SSH does. When a client connects to a SSH or SFTP server, the server will respond with that key’s public key and fingerprint. The client will inspect that fingerprint to see if it matches up precisely with the fingerprint received the last time it connected to that host. If there is a mismatch, there is a chance of a man-in-the-middle attack. So how does the client know if they are connected to the legitimate host the first time? Unlike TLS (the protocol used with https:\/\/ when you see the little padlock), there is no trusted certificate chain that can be inspected to see if the host can be trusted.* (While there are SSH implementations that support X.509 certificates, they are not commonly used). If you want to verify that you are connecting to the real CFTP when following the directions below, make sure the key fingerprint returned when connecting to cftp.fullcontact.com matches one of the hashes below:<\/span><\/p>\n MD5 : 14:b8:56:4f:f4:2b:3c:fe:d7:4c:8c:b1:85:05:cc:1e<\/span> Confidentiality in transit refers to how the information transmitted between Sally and <\/span>ftp.acme.inc<\/span> can remain secret even if Evil Bill has access to the network.\u00a0<\/span><\/p>\n The original FTP does not provide confidentiality in transit and allows Evil Bill to see the username\/password and all of the data transmitted between Sally and <\/span>ftp.acme.inc<\/span>.\u00a0<\/span><\/p>\n After server identity is confirmed in SFTP, a secure channel is negotiated and established between the client\/server using a symmetric key to encrypt all future communications. This allows all of the data sent between Sally and <\/span>ftp.data.inc<\/span> to remain secret even if there is a lurking Evil Bill.\u00a0<\/span><\/p>\n In addition to secrecy, the secure channel also provides integrity by using a standard hashing algorithm. This lets both parties know that the data they are receiving has not been modified somewhere in transit.\u00a0<\/span><\/p>\n Since CFTP is an implementation of SFTP and uses the same secure channels, CFTP has the same secrecy and integrity guarantees as SFTP.<\/span><\/p>\n While keeping Evil Bill from reading your data during transmission is important, keeping your data secret after it is stored on disk is also important. By default, both FTP and SFTP store their files to disk exactly as they received them. While it is possible to encrypt the files using a tool like PGP before sending it to an SFTP server, clients often send their data as is (in unencrypted plain text). Even when files are encrypted using PGP, it is very possible that when an individual decrypts the files to process them, the raw decrypted versions may remain on disk much longer than intended. That is unless an organization has strict controls and policies around handling files.\u00a0<\/span><\/p>\n CFTP approaches this differently and provides confidentiality at rest by default without forcing the client to encrypt their data beforehand using a tool like PGP.\u00a0\u00a0<\/span><\/p>\n As a preview, here is what encrypting files on the command line using both openssl and pgp look like. Commands in bold below are typed by the user where non bold lines represent output on the terminal.<\/span><\/p>\n As the raw bytes are received by CFTP, they are buffered in memory and sent in chunks to an internal service called Ciphervisor. Ciphervisor is designed to look up the client specific symmetric key for the account, perform symmetric encryption (AES-128-CTR) using that key and store the encrypted data to an AWS S3 bucket (which also has bucket encryption applied). Keeping the raw data in memory as you encrypt it reduces the risk to that data if an attacker was able to gain access to the server. When paired with the fact that different encryption keys are used for each account the risk that the attacker gains access to your raw unencrypted data is greatly reduced.<\/span><\/p>\n In this way, even if an internal actor is able to gain access to an encrypted file or object in S3, they are unable to read it without sending the data through Ciphervisor and requesting that it be decrypted for them.\u00a0<\/span><\/p>\n In systems like FTP and SFTP, auditing typically refers to the process of logging and recording which users have logged in and which actions have taken place. While FTP and SFTP do not provide comprehensive logging out of the box, they can be configured to log file-specific actions taken by each user. With FTP and SFTP, once files are accessed directly on the server by internal users, this log trail is usually lost. And even worse, if Evil Bill ever got into the machine, he could change the logs or even delete them!<\/span><\/p>\n With CFTP, any time the data needs to be read, the user must first request it be decrypted through Ciphervisor. This decrypt action is logged and audited. Unlike SFTP, access to files continues to be logged by default even after they are uploaded to the server.<\/span><\/p>\n Follow these steps to start using CFTP to send and receive data to\/from FullContact.<\/span><\/p>\n Note:<\/span> Use your full email address for username e.g. <\/span><\/i>john.doe@fullcontact.com<\/span><\/i><\/p>\n When transferring your business’s most sensitive and valuable data, you want to know it’s being done securely and handled with care by those processing it. The primary security objectives you need to consider during any data transfer are:\u00a0<\/span><\/p>\n While it’s possible to achieve all of the above with SFTP and a few manual solutions, using CipherFTP (CFTP) is easy to accomplish all of them by default. More technical users can upload files to CFTP using a command line or GUI interfaces like CyberDuck or Filezilla. In contrast, users who just want a quick way to securely upload smaller files can take advantage of the new Files interface built into the FullContact <\/span>\n
Authentication<\/h3>\n
\n
![\"\"](\"https:\/\/www.fullcontact.com\/wp-content\/uploads\/2021\/10\/image1.png\")
<\/li>\n<\/ul>\n\n
![\"\"](\"https:\/\/www.fullcontact.com\/wp-content\/uploads\/2021\/10\/image9.png\")
<\/p>\n![\"\"](\"https:\/\/www.fullcontact.com\/wp-content\/uploads\/2021\/10\/image10.png\")
<\/p>\n![\"\"](\"https:\/\/www.fullcontact.com\/wp-content\/uploads\/2021\/10\/image5.png\")
<\/p>\nServer Identity and Trust<\/h3>\n
![\"\"](\"https:\/\/www.fullcontact.com\/wp-content\/uploads\/2021\/10\/image8.png\")
<\/p>\n
\n<\/span>SHA256 : v1oW8uKDdtol9vXlGyTMdcjxRnKH9t9ofjxTNWCaHj<\/span><\/p>\nConfidentiality in Transit<\/h3>\n
![\"\"](\"https:\/\/www.fullcontact.com\/wp-content\/uploads\/2021\/10\/image2.png\")
<\/p>\n![\"\"](\"https:\/\/www.fullcontact.com\/wp-content\/uploads\/2021\/10\/image3.png\")
<\/p>\nConfidentiality at Rest<\/h3>\n
OpenSSL Symmetric Encryption Example<\/i><\/b>\r\necho \"hi there\" > file.txt<\/b>\r\nopenssl aes-256-cbc -e -in file.txt -out file.txt.enc<\/b>\r\nenter aes-256-cbc encryption password:<\/span>\r\nopenssl aes-256-cbc -d -in file.txt.enc<\/b>\r\nenter aes-256-cbc decryption password:<\/span>\r\nhi there<\/span>\r\n\r\nOpenSSL Asymmetric Encryption Example (using public\/private keys)<\/i><\/b>\r\n# generate private and public key<\/span>\r\nopenssl genpkey -algorithm RSA -out private_key.pem -pkeyopt rsa_keygen_bits:2048<\/b>\r\nopenssl rsa -pubout -in private_key.pem -out public_key.pem<\/b>\r\n# encrypt file using public key<\/span>\r\nopenssl rsautl -encrypt -inkey public_key.pem -pubin -in file.txt file.txt.asym.enc<\/b>\r\n# decrypt file using private key<\/span>\r\nopenssl rsautl -decrypt -inkey private_key.pem -in file.txt.asym.enc -out file.txt.dec<\/b>\r\n\r\nPGP Asymmetric Encryption Example (using public\/private keys)<\/i><\/b>\r\n\r\ngpg --full-generate-key<\/b>\r\ngpg (GnuPG) 2.2.23; Copyright (C) 2020 Free Software Foundation, Inc.<\/span>\r\nThis is free software: you are free to change and redistribute it.<\/span>\r\nThere is NO WARRANTY, to the extent permitted by law.<\/span>\r\nPlease select what kind of key you want:<\/span>\r\n\u00a0\u00a0\u00a0(1) RSA and RSA (default)<\/span>\r\n\u00a0\u00a0\u00a0(2) DSA and Elgamal<\/span>\r\n\u00a0\u00a0\u00a0(3) DSA (sign only)<\/span>\r\n\u00a0\u00a0\u00a0(4) RSA (sign only)<\/span>\r\n\u00a0\u00a0(14) Existing key from card<\/span>\r\nYour selection? <\/span>1<\/b>\r\nRSA keys may be between 1024 and 4096 bits long.<\/span>\r\nWhat keysize do you want? (3072) <\/span>4096<\/b>\r\nRequested keysize is 4096 bits<\/span>\r\nPlease specify how long the key should be valid.<\/span>\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a00 = key does not expire<\/span>\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<n>\u00a0 = key expires in n days<\/span>\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<n>w = key expires in n weeks<\/span>\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<n>m = key expires in n months<\/span>\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<n>y = key expires in n years<\/span>\r\nKey is valid for? (0) <\/span>5y<\/b>\r\nKey expires at Sun Sep 27 11:51:02 2026 MDT<\/span>\r\nIs this correct? (y\/N) <\/span>y<\/b>\r\n\r\nGnuPG needs to construct a user ID to identify your key.<\/span>\r\n\r\nReal name: <\/span>cftpblogexample<\/b>\r\nEmail address: <\/span>cftpblogexample@fullcontact.com<\/b>\r\nComment: <\/span>this is an example for a blog<\/b>\r\nYou selected this USER-ID:<\/span>\r\n\u00a0\u00a0\u00a0\u00a0\"cftpblogexample (this is an example for a blog) <cftpblogexample@fullcontact.com>\"<\/span>\r\nChange (N)ame, (C)omment, (E)mail or (O)kay\/(Q)uit? <\/span>O<\/b>\r\nWe need to generate a lot of random bytes. It is a good idea to perform<\/span>\r\nsome other action (type on the keyboard, move the mouse, utilize the<\/span>\r\ndisks) during the prime generation; this gives the random number<\/span>\r\ngenerator a better chance to gain enough entropy.<\/span>\r\nWe need to generate a lot of random bytes. It is a good idea to perform<\/span>\r\nsome other action (type on the keyboard, move the mouse, utilize the<\/span>\r\ndisks) during the prime generation; this gives the random number<\/span>\r\ngenerator a better chance to gain enough entropy.<\/span>\r\ngpg: key 37F3D947A60BC20C marked as ultimately trusted<\/span>\r\ngpg: revocation certificate stored as '\/Users\/user\/.gnupg\/openpgp-revocs.d\/ED3E9BCD84C8720C8653D42737F3D947A60BC20C.rev'<\/span>\r\npublic and secret key created and signed.<\/span>\r\n\r\npub \u00a0 rsa4096 2021-09-28 [SC] [expires: 2026-09-27]<\/span>\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0ED3E9BCD84C8720C8653D42737F3D947A60BC20C<\/span>\r\nuid\u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 cftpblogexample (this is an example for a blog) <cftpblogexample@fullcontact.com><\/span>\r\nsub \u00a0 rsa4096 2021-09-28 [E] [expires: 2026-09-27]<\/span>\r\n\r\nuser@FullContact-user-MBP 20210927 % gpg --list-keys<\/span>\r\ngpg: checking the trustdb<\/span>\r\ngpg: marginals needed: 3\u00a0 completes needed: 1\u00a0 trust model: pgp<\/span>\r\ngpg: depth: 0\u00a0 valid: \u00a0 2\u00a0 signed: \u00a0 0\u00a0 trust: 0-, 0q, 0n, 0m, 0f, 2u<\/span>\r\ngpg: next trustdb check due at 2022-01-26<\/span>\r\n\/Users\/user\/.gnupg\/pubring.kbx<\/span>\r\n----------------------------------------<\/span>\r\n\r\ngpg --export --armor cftpblogexample > cftpblogexample_public.asc<\/b>\r\ngpg --import cftpblogexample_public.asc<\/b>\r\ngpg --encrypt --recipient cftpblogexample file.txt<\/b>\r\ngpg --decrypt file.txt.gpg<\/b>\r\ngpg: encrypted with 4096-bit RSA key, ID 464A25175D1FC7F2, created 2021-09-28<\/span>\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\"cftpblogexample (this is an example for a blog) <cftpblogexample@fullcontact.com>\"<\/span>\r\nhi there<\/span><\/pre>\n![\"\"](\"https:\/\/www.fullcontact.com\/wp-content\/uploads\/2021\/10\/image4.png\")
<\/p>\nAuditing<\/h3>\n
![\"\"](\"https:\/\/www.fullcontact.com\/wp-content\/uploads\/2021\/10\/image13.png\")
<\/p>\nGetting started with CFTP<\/h3>\n
\n
\n
![\"\"](\"https:\/\/www.fullcontact.com\/wp-content\/uploads\/2021\/10\/image12.png\")
<\/span><\/li>\n![\"\"](\"https:\/\/www.fullcontact.com\/wp-content\/uploads\/2021\/10\/image11.png\")
<\/span><\/a><\/li>\n<\/ol>\n<\/li>\n\n
\n
\n<\/span>MD5 : 14:b8:56:4f:f4:2b:3c:fe:d7:4c:8c:b1:85:05:cc:1e<\/span>
\n<\/span>SHA256 : v1oW8uKDdtol9vXlGyTMdcjxRnKH9t9ofjxTNWCaHj<\/span><\/li>\n<\/ol>\n![\"\"](\"https:\/\/www.fullcontact.com\/wp-content\/uploads\/2021\/10\/image7.png\")
<\/p>\n![\"\"](\"https:\/\/www.fullcontact.com\/wp-content\/uploads\/2021\/10\/cyberduck.gif\")
If you are unable to use the command line SFTP or user interface like Cyberduck you can send a request to your sales or account representative to enable the web CFTP interface built into <\/span>Platform<\/span><\/a> (this is a new beta feature). Once this feature is enabled on your account you will be able to browse and upload small files (<= 1GB) by dragging and dropping them to your browser.<\/span><\/p>\n![\"\"](\"https:\/\/www.fullcontact.com\/wp-content\/uploads\/2021\/10\/cftpplatform.gif\")
<\/p>\nSummary<\/h3>\n
\n