Write console command to re-calculate counter aggregate value #47

New Issue

saavagebueno · 2025-11-20T04:15:38-05:00

saavagebueno commented

2025-11-20 04:15:38 -05:00

Originally created by @antonkomarev on GitHub (Sep 3, 2023).

After fix (by @Brikaa) of counter reset because of the race condition (#66) we need to create console command which will re-calculate correct counter values. It should count lines in storage/{username}-views file and store them to storage/{username}-views-count.

Something like:

cat storage/{username}-views | wc -l | tr -d ' ' > storage/{username}-views-count

But we should iterate thru all the usernames in the storage directory. There are many counter files, so ls -la is hanging for a long time.

One more thing to ensure: we need to make -1 from wc -l, because of the empty line at the end of file.

Originally created by @antonkomarev on GitHub (Sep 3, 2023). After fix (by @Brikaa) of counter reset because of the race condition (#66) we need to create console command which will re-calculate correct counter values. It should count lines in `storage/{username}-views` file and store them to `storage/{username}-views-count`. Something like: ```sh cat storage/{username}-views | wc -l | tr -d ' ' > storage/{username}-views-count ``` But we should iterate thru all the usernames in the storage directory. There are many counter files, so `ls -la` is hanging for a long time. One more thing to ensure: we need to make `-1` from `wc -l`, because of the empty line at the end of file.

saavagebueno commented

2025-11-20 04:15:38 -05:00

@antonkomarev commented on GitHub (Sep 3, 2023):

It looks like this command can compute correct value:

awk 'END {print NR-1}' storage/{username}-views > storage/{username}-views-count

@antonkomarev commented on GitHub (Sep 3, 2023): It looks like this command can compute correct value: ```shell awk 'END {print NR-1}' storage/{username}-views > storage/{username}-views-count ```

saavagebueno commented

2025-11-20 04:15:39 -05:00

@Brikaa commented on GitHub (Sep 3, 2023):

I believe wc -l does not count the new line at the end of the file

$ echo -e "hello\nthere" > test.txt && wc -l test.txt
2 test.txt

@Brikaa commented on GitHub (Sep 3, 2023): I believe `wc -l` does not count the new line at the end of the file ```bash $ echo -e "hello\nthere" > test.txt && wc -l test.txt 2 test.txt ```

saavagebueno commented

2025-11-20 04:15:39 -05:00

@antonkomarev commented on GitHub (Sep 3, 2023):

Try

echo -e "hello\nthere\n" > test.txt && wc -l test.txt

@antonkomarev commented on GitHub (Sep 3, 2023): Try ```bash echo -e "hello\nthere\n" > test.txt && wc -l test.txt ```

saavagebueno commented

2025-11-20 04:15:39 -05:00

@Brikaa commented on GitHub (Sep 3, 2023):

That puts two new lines at the end of the file.

@Brikaa commented on GitHub (Sep 3, 2023): That puts two new lines at the end of the file.

saavagebueno commented

2025-11-20 04:15:40 -05:00

@antonkomarev commented on GitHub (Sep 3, 2023):

That's exactly how views are stored. We have a blank line on the end of the file.

@antonkomarev commented on GitHub (Sep 3, 2023): That's exactly how views are stored. We have a blank line on the end of the file.

saavagebueno commented

2025-11-20 04:15:40 -05:00

@Brikaa commented on GitHub (Sep 3, 2023):

Yes, but your example puts two blank lines at the end of the file.

@Brikaa commented on GitHub (Sep 3, 2023): Yes, but your example puts two blank lines at the end of the file. ![image](https://github.com/antonkomarev/github-profile-views-counter/assets/40431124/18a1bf05-a0b5-4b36-b267-796448525658)

saavagebueno commented

2025-11-20 04:15:40 -05:00

@Brikaa commented on GitHub (Sep 15, 2023):

Does the following script using Perl's readdir() for listing the directory content hang? It shouldn't try to read the entire directory content at once.

my $dir = "/app/storage";
opendir(my $dh, $dir) || die "Can't open $dir: $!";
while (my $f = readdir $dh) {
  print "$dir/$f\n";
}
closedir $dh;

Save it as filename.pl and run in the container it using:

perl filename.pl

If it doesn't we can build upon that.

If it hangs, we can consider fixing the user on each HTTP request. We can have a flag that determines if a certain username is fixed, and on each request we check if the flag exists. If it doesn't, we fix the user and create the flag.

@Brikaa commented on GitHub (Sep 15, 2023): Does the following script using [Perl's readdir()](https://perldoc.perl.org/functions/readdir) for listing the directory content hang? It shouldn't try to read the entire directory content at once. ```perl my $dir = "/app/storage"; opendir(my $dh, $dir) || die "Can't open $dir: $!"; while (my $f = readdir $dh) { print "$dir/$f\n"; } closedir $dh; ``` Save it as `filename.pl` and run in the container it using: ```bash perl filename.pl ``` If it doesn't we can build upon that. If it hangs, we can consider fixing the user on each HTTP request. We can have a flag that determines if a certain username is fixed, and on each request we check if the flag exists. If it doesn't, we fix the user and create the flag.

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: starred/github-profile-views-counter#47