How to replace text randomly from file?

Question

How can I randomly replace specific strings in one text file with strings from another file? For example:

file1.txt(file has more than 200 lines):
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]

file2.txt(file has 10-20 lines):
@adress1.com
@adress2.com
@adress3.com
@adress4.com
@adress5.com

output.txt:
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]

Answer 1

You could implement this algorithm:

• Load the content of file2.txt to an array
• For each line in file1.txt:
  • Extract the name part
  • Get a random address
  • Print the output correctly formatted

Like this:

mapfile -t addresses < file2.txt
while IFS='' read -r orig || [[ -n "$orig" ]]; do
    ((index = RANDOM % ${#addresses[@]}))
    name=${orig%%@*}
    echo "$name${addresses[index]}"
done < file1.txt

(Special thanks to @GlennJackman and @dessert for the improvements.)

Answer 2

If you really want a random selection, then here's one way using awk:

awk '
  BEGIN{FS="@"; OFS=""} 
  NR==FNR{a[NR]=$0; n++; next} 
  {$2=a[int(1 + n * rand())]; print}
' file2.txt file1.txt
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]

OTOH if you want a random permutation of the addresses, I'd suggest something like

paste -d '' <(cut -d'@' -f1 file1.txt) <(sort -R file2.txt)
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]

Answer 3

You could use shuf (you might need to sudo apt install shuf) to shuffle the lines of the second file and then use them to replace:

$ awk -F'@' 'NR==FNR{a[NR]=$1;next}{print a[FNR]"@"$2} ' file1 <(shuf file2)
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]

shuf simply randomizes the order of its input lines. The awk command there will first read all of file1 (NR==FNR will only be true while the first file is being read), and saves the second field (fields are defined by @, so this is the domain) in the associative array a whose values are the domains and whose keys are the line numbers. Then, when we get to the next file, it will simply print whatever was stored in a for this line number, along with what's in file 2 for the same line number.

Note that this assumes both files have exactly the same number of lines and isn't actually being "random", since it will not allow anything to be repeated. But that looks like what you wanted to ask for.

Answer 4

Python 2.7 and 3 solution

This solution replaces the first occurrence of a single arbitrary given string (the “needle”) in every line of the input file with a string each time chosen randomly from the set of lines of the replacements string list.

#!/usr/bin/python
from __future__ import print_function
import sys, random

needle = sys.argv[1]

if sys.argv[2] == '-':
    f_replacements = sys.stdin
else:
    f_replacements = open(sys.argv[2])
with f_replacements:
    replacements = [l.rstrip('\n') for l in f_replacements]
if not replacements:
    raise ValueError('No replacement strings given')

if len(sys.argv) <= 3 or sys.argv[3] == '-':
    f_in = sys.stdin
else:
    f_in = open(sys.argv[3])
with f_in:
    for s in f_in:
        rep = replacements[random.randrange(len(replacements))]
        print(s.rstrip('\n').replace(needle, rep, 1))

It should be almost trivial to anchor the needle to the beginning or end of the string or to use regular expressions altogether.

Usage

python replace-random.py NEEDLE REPLACEMENTS-FILE [INPUT-FILE]

Example:

python replace-random.py '@address.com' file2.txt file1.txt

or

python replace-random.py '@address.com' file2.txt < file1.txt

Answer 5

Here's a perl way:

#!/usr/bin/perl
use warnings;
use strict;
use Tie::File;

tie my @file1,'Tie::File','file1.txt' or die "Can't open file1.txt\n";
tie my @file2,'Tie::File','file2.txt' or die "Can't open file2.txt\n";

for my $file_index (0..$#file1) {
   my $suffix = $file2[int(rand($#file2+1))];
   $file1[$file_index] =~ s/@.*$/$suffix/;
}

untie @file1;
untie @file2;

Answer 6

Another bash solution. It uses bash built-in string replacement feature. It also assumes file2.txt contains the replacement strings only. If not they can be first filtered using grep -o <replace> file2.txt

With shuf

#search string
Search="@address.com"
for lines in $(grep $Search file1.txt)
do 
    echo ${lines/$Search/$(shuf file2.txt -n 1)} 
done

Without shuf (almost pure bash)

Here we have to create a function first that mimics shuf like so

bshuf () 
{ 
    nlines=$(( $(wc -l < $1) + 1))
    rand=0
    while [ "$rand" -eq 0 ]; do
        rand=$(( $RANDOM % nlines ))
    done
    echo $(head -n $rand $1 | tail -1)
}

Then it is similar

for lines in $(grep $Search file1.txt) 
do 
    echo ${lines/$Search/$(bshuf file2.txt)}
done

Test:

$ for lines in $(grep $Search file1.txt); do echo ${lines/$Search/$(bshuf file2.txt)} ; done
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
$