Comparison of Easynews and Astraweb NNTP Servers

[Jack Tesla]

Let¹s compare two NNTP News Servers.

Server #1 is Easynews, which currently claims 1800 days of retention.

Server #2 is Astraweb, which currently claims 2024 days of retention.

Astraweb has 224 more days of retention than Easynews.

But, for the purpose of this comparison, I did not include any Astraweb
articles older than 1800 days (3/27/09).

This is a fair comparison of Easynews vs. Astraweb over the same 1800
day period (3/27/09 - 3/1/14).

I compared the article counts on Easynews and Astraweb for 20 different
groups. I picked the group that has the largest difference:
alt.binaries.pictures.katie.

Easynews article count: 11,142

Astraweb article count: 1,942,197

That¹s an enormous difference of 1,931,055 more articles on Astraweb.
I could understand a 10% difference (194,000). But a difference of
more than 1.9 million articles is astonishing.

For those people who are concerned about getting a ³full NNTP feed²,
you have the right to be concerned. I¹d like to hear the explanation
for this difference. If it happened for one group, who knows how many
other groups are affected?

[Espada]

Post by Jack Tesla
Let¹s compare two NNTP News Servers.
Server #1 is Easynews, which currently claims 1800 days of retention.
Server #2 is Astraweb, which currently claims 2024 days of retention.
Astraweb has 224 more days of retention than Easynews.
But, for the purpose of this comparison, I did not include any Astraweb
articles older than 1800 days (3/27/09).
This is a fair comparison of Easynews vs. Astraweb over the same 1800
day period (3/27/09 - 3/1/14).
I compared the article counts on Easynews and Astraweb for 20 different
alt.binaries.pictures.katie.
Easynews article count: 11,142
Astraweb article count: 1,942,197
That¹s an enormous difference of 1,931,055 more articles on Astraweb.
I could understand a 10% difference (194,000). But a difference of
more than 1.9 million articles is astonishing.
For those people who are concerned about getting a ³full NNTP feed²,
you have the right to be concerned. I¹d like to hear the explanation
for this difference. If it happened for one group, who knows how many
other groups are affected?

One cause of missing files is removal of the article due to copyright
infringement. Unfortunately, in this case we cannot help you.

If you believe there may be another issue, please let us know what
groups and Message IDs of the articles you are having problems with so
that we can take a look at them. Here is an example Message ID -
<***@yoyo.com>. Also, please let us know
the server address you are using.

You can email this information to ***@easynews.com.

[Jack Tesla]

Post by Espada
One cause of missing files is removal of the article due to copyright
infringement. Unfortunately, in this case we cannot help you.
If you believe there may be another issue, please let us know what
groups and Message IDs of the articles you are having problems with so
that we can take a look at them. Here is an example Message ID -
the server address you are using.

The copyright deletion rate for image newsgroups is less than 10%.

For the sake of this discussion, let's assume that the copyright
deletion rate is 50%. That still leaves a difference of:

971,098 - 11,142 = 959,956 articles

Copyright deletion does not explain the difference. For some reason,
more than one million articles are missing from Easynews.


Easynews NNTP server address: secure-us.news.easynews.com

[novadales]

Post by Jack Tesla

Post by Espada
One cause of missing files is removal of the article due to copyright
infringement. Unfortunately, in this case we cannot help you.
If you believe there may be another issue, please let us know what
groups and Message IDs of the articles you are having problems with so
that we can take a look at them. Here is an example Message ID -
the server address you are using.

The copyright deletion rate for image newsgroups is less than 10%.
For the sake of this discussion, let's assume that the copyright
971,098 - 11,142 = 959,956 articles
Copyright deletion does not explain the difference. For some reason,
more than one million articles are missing from Easynews.
Easynews NNTP server address: secure-us.news.easynews.com

quick question. why doesn't astraweb have to remove the same copyright files
as easynews? don't they?

[lkmcmillen]

Post by novadales

Post by Jack Tesla

Post by Espada
One cause of missing files is removal of the article due to copyright
infringement. Unfortunately, in this case we cannot help you.
If you believe there may be another issue, please let us know what
groups and Message IDs of the articles you are having problems with so
that we can take a look at them. Here is an example Message ID -
the server address you are using.

The copyright deletion rate for image newsgroups is less than 10%.
For the sake of this discussion, let's assume that the copyright
971,098 - 11,142 = 959,956 articles
Copyright deletion does not explain the difference. For some reason,
more than one million articles are missing from Easynews.
Easynews NNTP server address: secure-us.news.easynews.com

quick question. why doesn't astraweb have to remove the same copyright files
as easynews? don't they?

The discrepancies in the number of messages/files available from Easynews and other news
servers, particularly those of Astraweb and Giganews, isn't new information. Based on my
use and observations, there are huge chunks of messages/files missing from Easynews which
can be found on the servers of other usenet providers. But I've never had the time/desire
to determine exactly what messages files are missing or the reasons for why they are missing.

I've been using Newsbin Pro for as long as it first became available. When Easynews
first announced the availability of the us-secure-news-beta.easynews.com I reinstalled
Newsbin Pro, set it up for use with only Easynews and Astraweb and their
us-secure-news-beta.easynews.com and ssl-us.astraweb.com nntp servers, and retrieved all
available headers from both servers for the newsgroups I subscribe to. When I download a
message/file of interest I have no idea from which server the messaage/file is
downloaded. Newsbin downloads the message/file from whichever server has the message,
and the highest set priority. With the Easynews server set to Priority 2, and Astraweb
set to priority 1, making the Easynews the server with the highest priority which Newsbin
first tries to download messages/files from, Newsbin reports that 73% of all of the
messages/files which I've downloaded from usenet since January 1, 2014 have come from Astraweb.

Why do I continue to use and support Easynews? Because of their http news servers and the
Global Search engine which provide a more comprehensive display of what's available than
does Newsbin Pro. It' my understanding that the Global Search engines ability to display
the contents of container files (i.e. Zip and RAR) is what makes Easynews the usenet service
provider of choice for the agents and employees of the owners of the copyrighted books,
movies, music, and software illegally posted to usenet, to search for and identify the posting
of such materials, and using the associated displayed information to compose and submit
specific written requests for the removal of such materials.

FWIW, I suspect that Easynews also employs scripts, which employ the use of a privately
maintained database of file signatures, to automatically identify and remove copyrighted
material from it's usenet servers, presumably based on some form of blanket request from
the owners of copyrighted materials. It's known that such scripts and file signature
databases are employed to identify and remove pornographic images and videos of children(*).

It's the "enhanced" indexing, and the long term storage of usenet messages, which are hastening
the death of usenet. As with America's Old/Wild West, the domain of alt (Anarchists, Liars and
Thieves) is being tamed and will eventually become little more than a memory.

Happy Trails to all.

* Financial donations to support the National Center for Missing and Exploited Children
[https://secure.missingkids.com/home] can be made online using a credit card at
https://secure.missingkids.com/Donate/Online

[Espada]

Post by Jack Tesla

Post by Espada
One cause of missing files is removal of the article due to copyright
infringement. Unfortunately, in this case we cannot help you.
If you believe there may be another issue, please let us know what
groups and Message IDs of the articles you are having problems with so
that we can take a look at them. Here is an example Message ID -
the server address you are using.

The copyright deletion rate for image newsgroups is less than 10%.
For the sake of this discussion, let's assume that the copyright
971,098 - 11,142 = 959,956 articles
Copyright deletion does not explain the difference. For some reason,
more than one million articles are missing from Easynews.
Easynews NNTP server address: secure-us.news.easynews.com

Thanks for that. I'll also need some of the MIDs that are appearing on
Astraweb's side, and not ours.

[bones39]

Post by Espada

Post by Jack Tesla

Post by Espada
One cause of missing files is removal of the article due to copyright
infringement. Unfortunately, in this case we cannot help you.
If you believe there may be another issue, please let us know what
groups and Message IDs of the articles you are having problems with so
that we can take a look at them. Here is an example Message ID -
the server address you are using.

The copyright deletion rate for image newsgroups is less than 10%.
For the sake of this discussion, let's assume that the copyright
971,098 - 11,142 = 959,956 articles
Copyright deletion does not explain the difference. For some reason,
more than one million articles are missing from Easynews.
Easynews NNTP server address: secure-us.news.easynews.com

Thanks for that. I'll also need some of the MIDs that are appearing on
Astraweb's side, and not ours.
--
Espada
Easynews Support Team
https://members.easynews.com/feedback/?a=Espada

This is a futile argument IMHO. Both are separate businesses. Everyone has the right to
run their business as they see fit. I have no problem with the file/group deletions by
easynews. I would just like the service they offer to run smoothly without so many day
to day issues. Speaking for myself of course I do not subscribe to two different news
service subscribers. That would be redundant IMHO. I've been with easynews for over a
decade and they have always offered an exclusive HTTP web interface which sets them
aside from their competition. If I were an individual who utilized a news reader then I
would do my comparison and then make a decision based upon my findings. So the real
question is WHY and individual would even subscribe to two different news services when
using the NTTP side of things since the usenet postings overall is just a fraction of
what it used to be four years ago.
IMHO a company that offers a HTTP web interface that provides instant images and
streaming video HAS TO BE ran differently in a more conservative and unfortunately
censoring way in order to avoid certain legal issues.

[Jack Tesla]

Post by bones39
This is a futile argument IMHO. Both are separate businesses.

The numbers I reported are real. But it's possible that Astraweb has
left a lot of "junk" articles that EN removed. It's also possible that
the Astraweb server has a corrupt numbering system for this group. This
could result in article counts that are much higher than what is
actually on the server.

Ultimately the comparison has problems and I've deciced not to continue
posting about it.

Thanks for the feedback.

[charles]

Post by Espada

Post by Jack Tesla

Post by Espada
One cause of missing files is removal of the article due to copyright
infringement. Unfortunately, in this case we cannot help you.
If you believe there may be another issue, please let us know what
groups and Message IDs of the articles you are having problems with so
that we can take a look at them. Here is an example Message ID -
the server address you are using.

The copyright deletion rate for image newsgroups is less than 10%.
For the sake of this discussion, let's assume that the copyright
971,098 - 11,142 = 959,956 articles
Copyright deletion does not explain the difference. For some reason,
more than one million articles are missing from Easynews.
Easynews NNTP server address: secure-us.news.easynews.com

Thanks for that. I'll also need some of the MIDs that are appearing on
Astraweb's side, and not ours.

Discussions like this cause me to wonder - why doesn't EN get a
subscription to Astraweb, then the techncal staff could look to see
what's different and report it to the administrative staff in the
manner they'd prefer. What would it cost, $10 or $20 a month?

Dumping the data gathering back onto the customer doesn't seem like a
sensible business practice. It should be sufficient for the customer
to report something amiss in a group or a collection of groups,
trouble diagnosis should be the responsibility of the service
provider.

That's what I think, anyway.

[DevilsPGD]

Post by charles
Discussions like this cause me to wonder - why doesn't EN get a
subscription to Astraweb, then the techncal staff could look to see
what's different and report it to the administrative staff in the
manner they'd prefer. What would it cost, $10 or $20 a month?

Back in the day, usenet providers often gave each other free NNTP
accounts when they had peering arrangements set up as it greatly
simplified troubleshooting to be able to peek at problematic accounts.

I'm not sure if that friendly relationship still exists though; I know
of at least one provider that was sucking articles off of an "unlimited"
account to back-fill their own server, that may have left a bad taste in
some mouths.

[Roger Zelazny]

Post by DevilsPGD
Back in the day, usenet providers often gave each other free NNTP
accounts when they had peering arrangements set up as it greatly
simplified troubleshooting to be able to peek at problematic accounts.
I'm not sure if that friendly relationship still exists though; I know
of at least one provider that was sucking articles off of an "unlimited"
account to back-fill their own server, that may have left a bad taste in
some mouths.

I started on Usenet in 1995. The first commercial server I subscribed
to was "Supernews" run by Jeremy Nixon. (Jeremy later got into trouble
for editing posts.) In 1998 I went to "Newscene". In 2001 I switched
to Easynews.

I remember the issue you mentioned about one provider using the
unlimited account to back-fill. That was discussed for a long time.

I doubt the old peering arrangements still exist. But it makes me
wonder: when EN went with the 1800-day retention, didn't they have to
do a huge amount of back-filling? (The older articles no longer existed
on their servers.) Where did all those older articles come from?

[DevilsPGD]

Post by Roger Zelazny

Post by DevilsPGD
Back in the day, usenet providers often gave each other free NNTP
accounts when they had peering arrangements set up as it greatly
simplified troubleshooting to be able to peek at problematic accounts.
I'm not sure if that friendly relationship still exists though; I know
of at least one provider that was sucking articles off of an "unlimited"
account to back-fill their own server, that may have left a bad taste in
some mouths.

I started on Usenet in 1995. The first commercial server I subscribed
to was "Supernews" run by Jeremy Nixon. (Jeremy later got into trouble
for editing posts.) In 1998 I went to "Newscene". In 2001 I switched
to Easynews.
I remember the issue you mentioned about one provider using the
unlimited account to back-fill. That was discussed for a long time.
I doubt the old peering arrangements still exist. But it makes me
wonder: when EN went with the 1800-day retention, didn't they have to
do a huge amount of back-filling? (The older articles no longer existed
on their servers.) Where did all those older articles come from?

The old peering arrangements still exist because without them, no
articles would flow between providers. However, the friendliness that
went with peering struggled some years ago.

As far as backfilling, Easynews is bigger than just Easynews, it's
likely that the articles existed elsewhere in the UNS family
(Newshostings, Eweka even, etc). There was a period where such articles
were available but the overviews (headers) were not, which probably
meant that Easynews's frontend servers knew how to access not only
Easynews backends, but also backends of their sister providers.

But that's just an educated guess.

[MeAtHome]

Post by Roger Zelazny

Post by DevilsPGD
Back in the day, usenet providers often gave each other free NNTP
accounts when they had peering arrangements set up as it greatly
simplified troubleshooting to be able to peek at problematic accounts.
I'm not sure if that friendly relationship still exists though; I know
of at least one provider that was sucking articles off of an "unlimited"
account to back-fill their own server, that may have left a bad taste in
some mouths.

I started on Usenet in 1995. The first commercial server I subscribed
to was "Supernews" run by Jeremy Nixon. (Jeremy later got into trouble
for editing posts.) In 1998 I went to "Newscene". In 2001 I switched
to Easynews.
I remember the issue you mentioned about one provider using the
unlimited account to back-fill. That was discussed for a long time.
I doubt the old peering arrangements still exist. But it makes me
wonder: when EN went with the 1800-day retention, didn't they have to
do a huge amount of back-filling? (The older articles no longer existed
on their servers.) Where did all those older articles come from?

I've been in Usenet as long as you have. I, too, started with
Supernews and remember Jeremy.
I never heard about him getting in trouble for editing posts. Where
did you hear that? And are you talking about editing posts in the
support group?
Just curious.

[Roger Zelazny]

Post by MeAtHome
I've been in Usenet as long as you have. I, too, started with
Supernews and remember Jeremy.
I never heard about him getting in trouble for editing posts. Where
did you hear that? And are you talking about editing posts in the
support group?
Just curious.

Do a Google search using all these words: supernews jeremy nixon

There are several links. This happened long after I was gone.

[Joe Heller]

Post by charles
Discussions like this cause me to wonder - why doesn't EN get a
subscription to Astraweb, then the techncal staff could look to see
what's different and report it to the administrative staff in the
manner they'd prefer. What would it cost, $10 or $20 a month?

You make a very good point. Two years ago I paid Astraweb a one-time
fee of $10 for 25 GB of bandwidth. I still have 4 GB left. Even if
there is no peering arrangement with Astraweb, $10 is nothing. Let one
of the support staff do it.

Post by charles
Dumping the data gathering back onto the customer doesn't seem like a
sensible business practice. It should be sufficient for the customer
to report something amiss in a group or a collection of groups,
trouble diagnosis should be the responsibility of the service
provider.

Another good point. If I were the Easynews NNTP Administrator, I would
investigate the article difference. The Administrator is getting paid
to do this. The customers, on the other hand, are paying for the
service. The operative words are "getting paid" and "paying for".
Somebody at EN needs to wake up and find the problem(s).

[Bill C]

Post by Joe Heller

Post by charles
Discussions like this cause me to wonder - why doesn't EN get a
subscription to Astraweb, then the techncal staff could look to see
what's different and report it to the administrative staff in the
manner they'd prefer. What would it cost, $10 or $20 a month?

You make a very good point. Two years ago I paid Astraweb a one-time
fee of $10 for 25 GB of bandwidth. I still have 4 GB left. Even if
there is no peering arrangement with Astraweb, $10 is nothing. Let one
of the support staff do it.

Post by charles
Dumping the data gathering back onto the customer doesn't seem like a
sensible business practice. It should be sufficient for the customer
to report something amiss in a group or a collection of groups,
trouble diagnosis should be the responsibility of the service
provider.

Hence- Probably one of the problems I'm experiencing with the vid/jpeg images in deeper
retention. I bet the files in expanded retention were pirated from another source and
thumbnailed at that time but when something went wrong easynews is now somewhat
bewildered at how to regenerate the thumbnails. I have not been able to pin down the
problem to a specific date range, format, poster...blah...blah ...blah and I think
easynews is unable to pin down the files in order to regenerate the thumbs again. This
is probably why both Jason and muZe have been giving me the run around for almost two
months on this matter with no definitive date on a fix.

Post by Joe Heller
Another good point. If I were the Easynews NNTP Administrator, I would
investigate the article difference. The Administrator is getting paid
to do this. The customers, on the other hand, are paying for the
service. The operative words are "getting paid" and "paying for".
Somebody at EN needs to wake up and find the problem(s).

[Joe Heller]

Post by Joe Heller
You make a very good point. Two years ago I paid Astraweb a one-time
fee of $10 for 25 GB of bandwidth.

To be accurate, I should have written: Two years ago I paid Astraweb a
one-time fee of $10 for 25 GB of data transfer. (Bandwidth has nothing
to do with it.) At this point in time, Astraweb also counts 20% of
header downloads as part of the total data transfer.

I guess I must have zoned out when I wrote the first response. I had
stayed awake for 2 full days waiting for a meaningful EN response to
the "Astraweb Controversy".

I guess at this point it's not going to happen unless EN customers do
all the work. Hey, maybe next we'll get to run the server. I nominate
"DevilsPGD" for the new NNTP administrator and "lkmcmillen" as
co-administrator.

[arturob]

Post by Jack Tesla
I compared the article counts on Easynews and Astraweb for 20 different
alt.binaries.pictures.katie.
Easynews article count: 11,142
Astraweb article count: 1,942,197

Perhaps one reason is that the group in question contains mainly softcore and semihardcore
image of children. Posting seems to have stopped about 1880 days ago and there are hardly any
new articles. Cant say I'm too sad about that

[Jack Tesla]

Post by arturob
Perhaps one reason is that the group in question contains mainly softcore and
semihardcore image of children. Posting seems to have stopped about
1880 days ago and there are hardly any new articles. Cant say I'm too
sad about that

When I did this comparison, I only looked at article counts. I never
looked at the headers. I thought "alt.binaries.pictures.katie" was a
group dedicated to images of the television personality, Katie Couric.
The content wasn't an issue.

Let's say that your description is correct: the newsgroup contains
mainly images of child pornography. According to you the posting
stopped around 1/1/09 (1880 days ago).

Since Easynews follows the directives of the Internet Watch Foundation,
this group should have been deleted from the EN servers at least 5
years ago. Why is the group still there?

[shawn]

Post by Jack Tesla

Post by arturob
Perhaps one reason is that the group in question contains mainly softcore and
semihardcore image of children. Posting seems to have stopped about
1880 days ago and there are hardly any new articles. Cant say I'm too
sad about that

When I did this comparison, I only looked at article counts. I never
looked at the headers. I thought "alt.binaries.pictures.katie" was a
group dedicated to images of the television personality, Katie Couric.
The content wasn't an issue.
Let's say that your description is correct: the newsgroup contains
mainly images of child pornography. According to you the posting
stopped around 1/1/09 (1880 days ago).
Since Easynews follows the directives of the Internet Watch Foundation,
this group should have been deleted from the EN servers at least 5
years ago. Why is the group still there?

No one complained about it to the IWF is the likely explanation. They
seem to be pretty quick to remove a group if it gets complaints.
Likely everyone posting in the newsgroup moved to another group.

[Jack Tesla]

Post by shawn
No one complained about it to the IWF is the likely explanation. They
seem to be pretty quick to remove a group if it gets complaints.
Likely everyone posting in the newsgroup moved to another group.

Last night, I logged into the EN secure NNTP server
(secure-us.news.easynews.com) and I searched the ³easynews.support²
newsgroup for all subjects that contain the word ³deleted².

Below is one of the articles I found that relates to this current
discussion.

On 2/14/2011, ³Joe Heller² posted an article with the subject: ³Revised
List od Deleted Newsgroups²

For those who love Message-ID¹s (Espada), here it is:
<140220111549272733%***@Redundancy.Dept>

The article lists 74 deleted Newsgroups. One of those deleted
newsgroups was: alt.binaries.pictures.katie.

So, as of 2/14/2011, this group was no longer on EN¹s servers. Based
on other posts here, it¹s understandable why it was removed.

I might add that ³shawn² posted 3 replies in this 2011 thread.

The question is: how did a group that was deleted in 2011 for child
porn, get added BACK to the EN servers sometime between 2/14/2011 and
1/1/2014?

There are other questionable newsgroups that were deleted in 2011 that
are also back on the EN servers. I¹ll leave it as an exercise for the
reader to determine which groups are back from the list of 74. :)

Is there some kind of ³appeals process² that got these deleted groups
back?

There must be some reason why EN has 1,931,055 fewer articles than
Astraweb for the same retention period for the same group. That was my
original question. I guess it was considered rhetorical.

[Spoon_Man]

On 2/14/2011, ³Joe Heller² posted an article with the subject: ³Revised
List od Deleted Newsgroups²

The correct subject is: "Revised List of Deleted Groups"

Thanks for your attention. I'm out.